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Preface 


Over thirty years have elapsed since the publication of Fox & Parker’s 1968 
text Chebyshev Polynomials in Numerical Analysis. This was preceded by 
Snyder’s brief but interesting 1966 text Chebyshev Methods in Numerical Ap- 
proximation. The only significant later publication on the subject is that by 
Rivlin (1974, revised and republished in 1990) — a fine exposition of the 
theoretical aspects of Chebyshev polynomials but mostly confined to these 
aspects. An up-to-date but broader treatment of Chebyshev polynomials is 
consequently long overdue, which we now aim to provide. 


The idea that there are really four kinds of Chebyshev polynomials, not 
just two, has strongly affected the content of this volume. Indeed, the prop- 
erties of the four kinds of polynomials lead to an extended range of results in 
many areas such as approximation, series expansions, interpolation, quadra- 
ture and integral equations, providing a spur to developing new methods. We 
do not claim the third- and fourth-kind polynomials as our own discovery, 
but we do claim to have followed close on the heels of Walter Gautschi in first 
adopting this nomenclature. 


Ordinary and partial differential equations are now major fields of applica- 
tion for Chebyshev polynomials and, indeed, there are now far more books on 
‘spectral methods’ — at least ten major works to our knowledge — than on 
Chebyshev polynomials per se. This makes it more difficult but less essential 
to discuss the full range of possible applications in this area, and here we have 
concentrated on some of the fundamental ideas. 


We are pleased with the range of topics that we have managed to in- 
clude. However, partly because each chapter concentrates on one subject 
area, we have inevitably left a great deal out — for instance: the updating of 
the Chebyshev—Padé table and Chebyshev rational approximation, Chebyshev 
approximation on small intervals, Faber polynomials on complex contours and 
Chebyshev (£..) polynomials on complex domains. 


For the sake of those meeting this subject for the first time, we have 
included a number of problems at the end of each chapter. Some of these, in 
the earlier chapters in particular, are quite elementary; others are invitations 
to fill in the details of working that we have omitted simply for the sake of 
brevity; yet others are more advanced problems calling for substantial time 
and effort. 


We have dedicated this book to the memory of two recently deceased col- 
leagues and friends, who have influenced us in the writing of this book. Geoff 
Hayes wrote (with Charles Clenshaw) the major paper on fitting bivariate 
polynomials to data lying on a family of parallel lines. Their algorithm retains 
its place in numerical libraries some thirty-seven years later; it exploits the 
idea that Chebyshev polynomials form a well-conditioned basis independent 
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of the spacing of data. Lev Brutman specialised in near-minimax approxima- 
tions and related topics and played a significant role in the development of 
this field. 


In conclusion, there are many to whom we owe thanks, of whom we can 
mention only a few. Among colleagues who helped us in various ways in the 
writing of this book (but should not be held responsible for it), we must name 
Graham Elliott, Ezio Venturino, William Smith, David Elliott, Tim Phillips 
and Nick Trefethen; for getting the book started and keeping it on course, Bill 
Morton and Elizabeth Johnston in England, Bob Stern, Jamie Sigal and others 
at CRC Press in the United States; for help with preparing the manuscript, 
Pam Moore and Andrew Crampton. We must finally thank our wives, Moya 
and Elizabeth, for the blind faith in which they have encouraged us to bring 
this work to completion, without evidence that it was ever going to get there. 


This book was typeset at Oxford University Computing Laboratory, using 
Lamport’s ATRX 2¢ package. 


John Mason 
David Handscomb 
April 2002 
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CHAPTER 1 


Definitions 


1.1 Preliminary remarks 


“Chebyshev olynomials are everywhere dense in numerical anal- 
p 
ysis.” 


This remark has been attributed to a number of distinguished mathematicians 
and numerical analysts. It may be due to Philip Davis, was certainly spoken 
by George Forsythe, and it is an appealing and apt remark. There is scarcely 
any area of numerical analysis where Chebyshev polynomials do not drop 
in like surprise visitors, and indeed there are now a number of subjects in 
which these polynomials take a significant position in modern developments 
— including orthogonal polynomials, polynomial approximation, numerical 
integration, and spectral methods for partial differential equations. 


However, there is a different slant that one can give to the quotation above, 
namely that by studying Chebyshev polynomials one is taken on a journey 
which leads into all areas of numerical analysis. This has certainly been our 
personal experience, and it means that the Chebyshev polynomials, far from 
being an esoteric and narrow subject, provide the student with an opportunity 
for a broad and unifying introduction to many areas of numerical analysis and 
mathematics. 


1.2 Trigonometric definitions and recurrences 


There are several kinds of Chebyshev polynomials. In particular we shall in- 
troduce the first and second kind polynomials T,,(a) and U,,(a), as well as 
a pair of related (Jacobi) polynomials V,,(”) and W,,(x), which we call the 
‘Chebyshev polynomials of the third and fourth kinds’; in addition we cover 
the shifted polynomials T(x), Uf (x), V;*(a) and W,*(x). We shall, however, 
only make a passing reference to ‘Chebyshev’s polynomial of a discrete vari- 
able’, referred to for example in Erdélyi et al. (1953, Section 10.23), since this 
last polynomial has somewhat different properties from the polynomials on 
which our main discussion is based. 


Some books and many articles use the expression ‘Chebyshev polynomial’ 
to refer exclusively to the Chebyshev polynomial T,,(a) of the first kind. In- 
deed this is by far the most important of the Chebyshev polynomials and, 
when no other qualification is given, the reader should assume that we too 
are referring to this polynomial. 


2 CHAPTER 1: DEFINITIONS 


Clearly some definition of Chebyshev polynomials is needed right away 
and, as we shall see as the book progresses, we are spoiled for a choice of 
definitions. However, what gives the various polynomials their power and 
relevance is their close relationship with the trigonometric functions ‘cosine’ 
and ‘sine’. We are all aware of the power of these functions and of their 
appearance in the description of all kinds of natural phenomena, and this 
must surely be the key to the versatility of the Chebyshev polynomials. We 
therefore use as our primary definitions these trigonometric relationships. 


1.2.1. The first-kind polynomial T;, 


Definition 1.1 The Chebyshev polynomial T,,(x) of the first kind is a poly- 
nomial in x of degree n, defined by the relation 


Tn(x) =cosn@ when x = cos0. (1.1) 


If the range of the variable «x is the interval [—1, 1], then the range of the 
corresponding variable @ can be taken as [0,7]. These ranges are traversed in 
opposite directions, since x = —1 corresponds to 0 = z and x = 1| corresponds 
to 6=0. 


It is well known (as a consequence of de Moivre’s Theorem) that cos n@ 
is a polynomial of degree n in cos@, and indeed we are familiar with the 
elementary formulae 


cos0?=1, cosl19@=cos6, cos2@=2cos?4—1, 
cos 30 = 4cos?@—3cos#, cos40 = 8cost@— 8cos?6+1, 
We may immediately deduce from (1.1), that the first few Chebyshev 


polynomials are 


To(2) = 1, T(z) =a, T2(2) = 2a? = 1, 


T3(x) = 4x? _ 32, T4(x) = Sart = Sa? + 1, (1.2) 


Coefficients of all polynomials T,,(#) up to degree n = 21 will be found in 
Tables C.2a, C.2b in Appendix C. 


In practice it is neither convenient nor efficient to work out each T,,(«) 
from first principles. Rather by combining the trigonometric identity 


cos n@ + cos(n — 2)@ = 2cos@ cos(n — 1)6 
with Definition 1.1, we obtain the fundamental recurrence relation 
Tn(£) = 2aT,-1(@) — Tn-2(x), n= 2,3,..., (1.3a) 
which together with the initial conditions 


To(t)=1, Ti(a)=2 (1.3b) 
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recursively generates all the polynomials {T;,(x)} very efficiently. 
It is easy to deduce from (1.3) that the leading coefficient (that of x”) in 


Tn(x) for n > 1 is double the leading coefficient in T,-1(#) and hence, by 
induction, is 2”~1. 


[\ po \ AN PA 
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Figure 1.1: T5(a) on range [—1, 1] Figure 1.2: cos5@ on range [0, 7] 


What does the polynomial T,,(z) look like, and how does a graph in the 
variable x compare with a graph of cosné@ in the variable 6? In Figures 1.1 
and 1.2 we show the respective graphs of T5(x) and cos5@. It will be noted 
that the shape of T5() on [—1, 1] is very similar to that of cos 56 on [0, a], and 
in particular both oscillate between six extrema of equal magnitudes (unity) 
and alternating signs. However, there are three key differences — firstly the 
polynomial T;(x) corresponds to cos 50 reversed (i.e., starting with a value of 
—1 and finishing with a value of +1); secondly the extrema of T5(a) at the end 
points x = +1 do not correspond to zero gradients (as they do for cos 50) but 
rather to rapid changes in the polynomial as a function of x; and thirdly the 
zeros and extrema of T5(x) are clustered towards the end points +1, whereas 
the zeros and extrema of cos 56 are equally spaced. 


The reader will recall that an even function f(x) is one for which 
f(x) = f(—2) for all x 
and an odd function f(x) is one for which 
f(x) = —f(—2) for all x. 


All even powers of x are even functions, and all odd powers of x are odd 
functions. Equations (1.2) suggest that T,,(x) is an even or odd function, 
involving only even or odd powers of x, according as n is even or odd. This 
may be deduced rigorously from (1.3a) by induction, the cases n = 0 and 
n = 1 being supplied by the initial conditions (1.3b). 


1.2.2. The second-kind polynomial U,, 


Definition 1.2 The Chebyshev polynomial U,,(x) of the second kind is a poly- 
nomial of degree n in x defined by 


U, (x) = sin(n+1)6/sin@ when x = cosé. (1.4) 


4 CHAPTER 1: DEFINITIONS 


The ranges of x and @ are the same as for T;,(x). 

Elementary formulae give 
sinl@=sin0@, sin20=2sin@ cos@, sin3@=sin@(4cos?@—1), 
sin 40 = sin 6 (8 cos® 0 — 4cos 8), 


sey 


so that we see that the ratio of sine functions (1.4) is indeed a polynomial in 
cos #, and we may immediately deduce that 


Uo(z) _ 1, Ui (x) = 2n, U2(x) = 4x? ~~ 1, 


U3(x) = 8x3 — 42, (1.5) 


Coefficients of all polynomials U,,(x) up to degree n = 21 will be found in 
Tables C.3a, C.3b in Appendix C. 


By combining the trigonometric identity 
sin(n + 1)@ + sin(n — 1)0 = 2cos@ sinnd 
with Definition 1.2, we find that U;,(x) satisfies the recurrence relation 
Uy (a) = 24Un_1(#) — Un_2(x), n= 2,3,..., (1.6a) 
which together with the initial conditions 
Uop(x) =1, Ui(a) = 2a (1.6b) 
provides an efficient procedure for generating the polynomials. 
A similar trigonometric identity 
sin(n + 1)@ — sin(n — 1)@ = 2sin@ cos nd 
leads us to a relationship 
U, (x2) — Un—2(x) = 2T, (x), n= 2,3,..., (1.7) 
between the polynomials of the first and second kinds. 
It is easy to deduce from (1.6) that the leading coefficient of x” in U,(«) 
is 2”. 
Note that the recurrence (1.6a) for {U,(a)} is identical in form to the 


recurrence (1.3a) for {T,(x)}. The different initial conditions [(1.6b) and 
(1.3b)] yield the different polynomial systems. 


In Figure 1.3 we show the graph of Us(x). It oscillates between six ex- 
trema, as does Ts() in Figure 1.1, but in the present case the extrema have 
magnitudes which are not equal, but increase monotonically from the centre 
towards the ends of the range. 


From (1.5) it is clear that the second-kind polynomial U,,(), like the first, 
is an even or odd function, involving only even or odd powers of x, according 
as n is even or odd. 
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Figure 1.3: Us(a#) on range [—1, 1] 


1.2.3. The third- and fourth-kind polynomials V,, and W,, (the air- 
foil polynomials) 


Two other families of polynomials V,, and W,, may be constructed, which are 
related to JT, and Uy, but which have trigonometric definitions involving the 
half angle 0/2 (where x = cos @ as before). These polynomials are sometimes! 
referred to as the ‘airfoil polynomials’, but Gautschi (1992) rather appropri- 
ately named them the ‘third- and fourth-kind Chebyshev polynomials’. First 
we define these polynomials trigonometrically, by a pair of relations parallel 
to (1.1) and (1.4) above for T,, and U,,. Again the ranges of x and 6 are the 
same as for T),(z). 


Definition 1.3 The Chebyshev polynomials V,(a) and W,,(a) of the third 
and fourth kinds are polynomials of degree n in x defined respectively by 


Vn(x) = cos(n + $)0/ cos $0 (1.8) 


and 


W,,(x) = sin(n + $)0/sin 38, (1.9) 


when x = cos 8@. 


To justify these definitions, we first observe that cos(n + 4)0 is an odd 
polynomial of degree 2n+1 in cos $0. Therefore the right-hand side of (1.8) is 
an even polynomial of degree 2n in cos 40, which is equivalent to a polynomial 
of degree n in cos? 40 = $(1 +cos@) and hence to a polynomial of degree n 
in cos@. Thus V,,(x) is indeed a polynomial of degree n in x. For example 


1+4)@ 4cos?46—3costé 
TCs ape ec A eS en ae 
cos 44 cos £6 2 
2 2 


We may readily show that 


Volz) =1, Vilz)=2xe-1, Va(x) = 4x? - 22-1, 
V3(x) = 803 — 4x? —4r +1, 
1See for example, Fromme & Golberg (1981). 


(1.10) 
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Similarly sin(n + 4)0 is an odd polynomial of degree 2n + 1 in sin 40. 
Therefore the right-hand side of (1.9) is an even polynomial of degree 2n in 
sin 36, which is equivalent to a polynomial of degree n in sin” $0 = 4(1—cos 6) 
and hence again to a polynomial of degree n in cos @. For example 

_ sin(Q1+3)0 — 3sin30—4sin® 30 


W. = 2 2 23 gin? 19 = 2cosO+1 = 2741. 
2 ease eT sin 40 m2 q 


We may readily show that 


Wo(z) =1, Wi(2) =2e+1, Wao(x) =4a2? + 22-1, (1.11) 
W3(x) = 8a? + 4a? — 4x —-1, 2 ; 


The polynomials V,,(2) and W,,() are, in fact, rescalings of two particular 


Jacobi polynomials P(x) with a = —4, 8 = 4 and vice versa. Explicitly 


) Vo (2) = 22" PO 2) (q), >") Wr (a) = Pr PME (2), 


Coefficients of all polynomials V,,(x) and W,,(x) up to degree n = 10 will 
be found in Table C.1 in Appendix C. 
These polynomials too may be efficiently generated by the use of a recur- 


rence relation. Since 


)9 + cos(n — 2+ $)@ = 2cos@ cos(n — 1+ 4)0 


cos(n + 4 


and 


sin(n + $)0 + sin(n — 2+ 4)0 = 2cos@ sin(n — 1+ $)0, 


it immediately follows that 


Vi(x) = 2aVp—-1(x) — Vn-2(x), n= 2,3,..., (1.12a) 

and 
Wr(a) = 2aW,-1(a) — Wr-2(x), n= 2,3,..., (1.12b) 

with 
Volx) =1, Vilx) = 22-1 (1.12c) 

and 
Wo(a)=1, W(x) = 2241. (1.12d) 


Thus V,,(#) and W,,(x) share precisely the same recurrence relation as 
T(x) and U,,(x), and their generation differs only in the prescription of the 
initial condition for n = 1. 


?See Chapter 22 of Abramowitz and Stegun’s Handbook of Mathematical Functions 
(1964). 
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It is immediately clear from (1.12) that both V,,(x) and W,,(«) are poly- 
nomials of degree n in x, in which all powers of x are present, and in which 
the leading coefficients (of x”) are equal to 2”. 


In Figure1.4 we show graphs of V5(”) and W5(a). They are exact inverted 
mirror images of one another, as will be proved in the next section (1.19). 


ee ee ei 


Figure 1.4: V5(a) and W5() on range [—1, 1] 


1.2.4 Connections between the four kinds of polynomial 


We already have a relationship (1.7) between the polynomials T,, and U,,. It 
remains to link V, and W,, to T,, and U,,. This may be done by introducing 
two auxiliary variables 

(1 —2)]? =sin 40. (1.13) 


u= [$(1 +2)]? =cosi6, t= [3 
Using (1.8) and (1.9) it immediately follows, from the definitions (1.1) and 
(1.4) of T,, and U;,, that 


Tp(t) = Ton(u), Un(c) = Fu ‘Uonqai(u), (1.14) 
Va(z) =u 'Tonsi(u), Wn(x) = Uon(u). (1.15) 


Thus T,,(z), Un(x), Vn(x), W(x) together form the first- and second-kind 
polynomials in u, weighted by u~+ in the case of odd degrees. Also (1.15) 
shows that V,,(x) and W,,(a) are directly related, respectively, to the first- and 
second-kind Chebyshev polynomials, so that the terminology of ‘Chebyshev 
polynomials of the third and fourth kind’ is justifiable. 


From the discussion above it can be seen that, if we wish to establish 
properties of V, and W,,, then we have two main options: we can start from the 
trigonometric definitions (1.8), (1.9) or we can attempt to exploit properties 
of T,, and U,, by using the links (1.14)—(1.15). 

Note that V, and W,, are neither even nor odd (unlike T,, and U,,). We 
have seen that the leading coefficient of x” is 2” in both V,, and W,, as it is 


in U,. This suggests a close link with U,. Indeed if we average the initial 
conditions (1.12c) and (1.12d) for V; and W, we obtain the initial condition 
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(1.6b) for U,, from which we can show that the average of V,, and W,, satisfies 
the recurrence (1.6a) subject to (1.6b) and therefore that for all n 


Un(x) = $[Va(z) + Wa(2)]. (1.16) 


The last result also follows directly from the trigonometric definitions (1.4), 
(1.8), (1.9) of Un, Van, Wn, since 


sin(n+1)0 _ sin(n + 5)0 cos 56 + cos(n + 5)4 sin 50 


sin 6 2sin 40 cos 40 


, [cos(n+4)0 — sin(n + 3)0 
? cos $0 sin $0 

Equation (1.16) is not the only link between the sets {V,,}, {W,} and 
{U,,}. Indeed, by using the trigonometric relations 


2sin $6 cos(n + $)9 = sin(n + 1) — sin nd, 


2cos $0 sin(n + $)@ = sin(n + 1)0 +sin nd 
and dividing through by sin @, we can deduce that 
Vila) = Un(x) — Un-i(2), (1.17) 
W(x) = Up(x) + Un-1(2). (1.18) 


Thus V,, and W,, may be very simply determined once {U,,} are available. 
Note that (1.17), (1.18) are confirmed in the formulae (1.5), (1.10), (1.11) 
and are consistent with (1.16) above. 


From the evenness/oddness of U;,(a) for n even/odd, we may immediately 
deduce from (1.17), (1.18) that 


W2(2) = Vn(—2) (n even); 


W,(z) =—Vn(-2) (n odd). (1.19) 


This means that the third- and fourth-kind polynomials essentially transform 
into each other if the range [—1, 1] of x is reversed, and it is therefore sufficient 
for us to study only one of these kinds of polynomial. 


Two further relationships that may be derived from the definitions are 
Vila) + Vn-1(2) = Wr(x) — Wr-1(2) = 2T,, (2). (1.20) 


If we were asked for a ‘pecking order’ of these four Chebyshev polynomials 
Tr, Un, Vn and Wy, then we would say that T), is clearly the most important 
and versatile. Moreover T;, generally leads to the simplest formulae, whereas 
results for the other polynomials may involve slight complications. However, 
all four polynomials have their role. For example, as we shall see, U,, is useful 
in numerical integration, while V,, and W,, can be useful in situations in which 
singularities occur at one end point (+1 or —1) but not at the other. 
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1.3. Shifted Chebyshev polynomials 


1.3.1 The shifted polynomials Ty, U7, V,*, W; 


Since the range [0,1] is quite often more convenient to use than the range 
[—1, 1], we sometimes map the independent variable x in [0,1] to the variable 
s in [—1, 1] by the transformation 


s=2x-—lorx= 5(1+s), 


and this leads to a shifted Chebyshev polynomial (of the first kind) T* (a) of 
degree n in x on [0,1] given by 


Te (x) = T,(s) = Tn(22 — 1). (1.21) 
Thus we have the polynomials 


T(x) =1, Th(z)=2r-1, T(x) = 827-82 +1, (1.22) 
T3 (x) = 322° — 482? + 182-1, ; 
From (1.21) and (1.3a), we may deduce the recurrence relation for T* in 


the form 
Ty (x) = 2(2@ — 1)Tp_1 (2) — Tr_2(2) (1.23a) 


with initial conditions 


T(x) =1, T(x) =2c-1. (1.23b) 


The polynomials T*(x) have a further special property, which derives from 
(1.1) and (1.21): 


Ton(x) = cos 2nd = cos n(20) = T,(cos 20) = T, (2a? — 1) = T* (2x?) 


so that 
Tula) = T(e"): (1.24) 


This property may readily be confirmed for the first few polynomials by com- 
paring the formulae (1.2) and (1.22). Thus T(x) is precisely Ton (./x), a 
higher degree Chebyshev polynomial in the square root of the argument, and 
relation (1.24) gives an important link between {T,,} and {7} which com- 
plements the shift relationship (1.21). Because of this property, Table C.2a 
in Appendix C, which gives coefficients of the polynomials T,,(a) up to de- 
gree n = 20 for even n, at the same time gives coefficients of the shifted 
polynomials T*(x) up to degree n = 10. 

It is of course possible to define Ty, like T;, and Uy, directly by a trigono- 
metric relation. Indeed, if we combine (1.1) and (1.24) we obtain 


T* (a) = cos 2n0 when a = cos” 0. (1.25) 


n 
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This relation might alternatively be rewritten, with 6 replaced by ¢/2, in the 
form 
T; (x) = cosn@ when x = cos” $/2 = $(1+ cos ¢). (1.26) 


Indeed the latter formula could be obtained directly from (1.21), by writing 


Tn(s) = cosn@ when s = cos ¢. 


Note that the shifted Chebyshev polynomial T* (x) is neither even nor odd, 
and indeed all powers of x from 1 = x° to x” appear in T*(x). The leading 
coefficient of «” in T(x) for n > 0 may be deduced from (1.23a), (1.23b) to 
be 227-1, 

Shifted polynomials U;, V,*, W,t of the second, third and fourth kinds may 
be defined in precisely analogous ways: 


UX (2) =U,(2a—-1), Vi (a) =Vr(2a-—1), W(x) = W,, (22 —1). (1.27) 


n n 


Links between U;, V,*, W; and the unstarred polynomials, analogous to (1.24) 
above, may readily be established. For example, using (1.4) and (1.27), 


sin #0 Ugn_1(x) = sin 2nd = sin n(26) = sin 26 U,,_1 (cos 20) 
= 2sin cos 6 Un_1(2a? — 1) = sind {22U7_,(x?)} 
and hence 


Uan—1(2) = 22U*_,(2”). (1.28) 


The corresponding relations for V;* and W; are slightly different in that 
they complement (1.24) and (1.28) by involving T>,_1 and U2,. Firstly, using 
(1.13), (1.15) and (1.27), 


V,*_1(u?) = Va_1(2u? — 1) = Va_i(x) = uw Top_i(u) 


nm 


and hence (replacing u by x) 
Toy, ~1 (0) = 2V,5 (2): (1.29) 
Similarly, 


* 1 (u2) = Wr_1(2u? — 1) = Wp_i(x) = Uon(u) 


n-1 
and hence (replacing u by x) 


Uon(x) = W(x"). (1.30) 


Because of the relationships (1.28)—(1.30), Tables C.3b, C.2b, C.3a in Ap- 
pendix C, which give coefficients of T,,(”) and U;,(x) up to degree n = 20, at 
the same time give the coefficients of the shifted polynomials U;*(x), V,* (a), 
W,* (a), respectively, up to degree n = 10. 
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1.3.2 Chebyshev polynomials for the general range |a, b] 


In the last section, the range [—1, 1] was adjusted to the range [0, 1] for conve- 
nience, and this corresponded to the use of the shifted Chebyshev polynomials 
Ty, Ux, Vx, Wx in place of T,,, Un, Vn, Wn respectively. More generally we 
may define Chebyshev polynomials appropriate to any given finite range [a, b] 
of x, by making this range correspond to the range [—1, 1] of a new variable 
s under the linear transformation 


Qn — b 
ga Zea tb) 


—— (1.31) 


The Chebyshev polynomials of the first kind appropriate to [a,b] are thus 
Tn(s), where s is given by (1.31), and similarly the second-, third- and fourth- 
kind polynomials appropriate to [a,b] are U;,(s), Vn(s), and W,,(s). 


EXAMPLE 1.1: The first-kind Chebyshev polynomial of degree three appropriate 
to the range [1, 4] of x is 


Qa—5\  ,(2e—5\° Pett Oe re 2 : 
r( ; ) =a 7 ) -3( ; ) = Flex — 240x? + 546x — 365). 


Note that in the special case [a,b] = [0,1], the transformation (1.31) be- 
comes s = 2% —1, and we obtain the shifted Chebyshev polynomials discussed 
in Section 1.3.1. 

Incidentally, the ‘Chebyshev Polynomials S;,(z) and C;,(a)’ tabulated by 
the National Bureau of Standards (NBS 1952) are no more than mappings of 
U,, and 2T,, to the range [a, b] = [—2,2]. Except for Co, these polynomials all 
have unit leading coefficient, but this appears to be their only recommending 
feature for practical purposes, and they have never caught on. 


1.4 Chebyshev polynomials of a complex variable 


We have chosen to define the polynomials T;,(a), Un(x), Vn(a) and W,,(2) 
with reference to the interval [—1,1]. However, their expressions as sums 
of powers of x can of course be evaluated for any real x, even though the 
substitution x = cos @ is not possible outside this interval. 


For x in the range [1, 00), we can make the alternative substitution 


x =coshO, (1.32) 


12 CHAPTER 1: DEFINITIONS 


with © in the range [0,00o), and it is easily verified that precisely the same 
polynomials (1.2), (1.5), (1.10) and (1.11) are generated by the relations 


Tnr(x) = coshn®, (1.33a) 
_ sinh(n + 1)0 
h +)0 
Va(z) = uaa Wem ee (1.33c) 
cosh 590 
sinh(n + 4)0 
Wh = —__ 1.33d 
(2) sinh 40 ( ) 


For « in the range (—oo, —1] we can make use of the odd or even parity of 
the Chebyshev polynomials to deduce from (1.33) that, for instance, 


Tn(x) = (—1)” coshnO 


where 
x =—coshe. 


It is easily shown from (1.33) that none of the four kinds of Chebyshev 
polynomials can have any zeros or turning points in the range [1,co). The 
same applies to the range (—co, —1]. This will later become obvious, since we 
shall show in Section 2.2 that T;,, Un, Vz, and W,, each have n real zeros in 
the interval [—1, 1], and a polynomial of degree n can have at most n zeros in 
all. 


The Chebyshev polynomial T,,(x) can be further extended into (or initially 
defined as) a polynomial T,,(z) of a complex variable z. Indeed Snyder (1966) 
and Trefethen (2000) both start from a complex variable in developing their 
expositions. 


1.4.1 Conformal mapping of a circle to and from an ellipse 


For convenience, we consider not only the variable z but a related complex 
variable w such that 
z= 3(wtw'). (1.34) 


Then, if w moves on the circle |w| = r (for r > 1) centred at the origin, 
we have 
w=re? =rcosé +irsind, (1.35) 


wt=r te? — rl cos6 —ir sind, (1.36) 


and so, from (1.34), 


z= acos@ + ibsiné (1.37) 
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where 
a=3(rt+r"), b=s(r—-r"’). (1.38) 


Hence z moves on the standard ellipse 
ed (1.39) 


centred at the origin, with major and minor semi-axes a, b given by (1.38). It 
is easy to verify from (1.38) that the eccentricity e of this ellipse is such that 


ae= Va? —b? = 1, 


and hence the ellipse has foci at z = +1. 


In the case r = 1, where w moves on the unit circle, we have b = 0 and 
the ellipse collapses into the real interval [—1,1]. However, z traverses the 
interval twice as w moves round the circle: from —1 to 1 as 8 moves from —7 
to 0, and from 1 to —1 as @ moves from 0 to 7. 


Figure 1.5: The circle |w| = r = 1.5 and its image in the z plane 


ie en 
ar ie al 


Figure 1.6: The circle |w| = 1 and its image in the z plane 


The standard circle (1.35) and ellipse (1.39) are shown in Figure 1.5, and 
the special case r = 1 is shown in Figure 1.6. See Henrici (1974-1986) for 
further discussions of this mapping. 
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From (1.34) we readily deduce that w satisfies 
w? —2wz+1=0, (1.40) 


a quadratic equation with two solutions 


wW=W1,W2 =ztvV2z2-1. (1.41) 


This means that the mapping from w to z is 2 to 1, with branch points at 
z= +1. It is convenient to define the complex square root Vz? — 1 so that it 
lies in the same quadrant as z (except for z on the real interval [—1, 1], along 
which the plane must be cut), and to choose the solution 


w=w,=2z+vV27-1, (1.42) 


so that |w| = |w:| > 1. Then w depends continuously on z along any path 
that does not intersect the interval [—1, 1], and it is easy to verify that 


Wy = w= 2-4/2? —1, (1.43) 


with |w2| < 1. 


If w; moves on |w;| = r, for r > 1, then we moves on |w2| = jwy "| = 
r—1 <1. Hence both of the concentric circles 


Cris {wilwl=r}, Cry i= {ws |wl =r} 


transform into the same ellipse defined by (1.37) or (1.39), namely 


2+ V2=1|=r}. (1.44) 


a {z: 


1.4.2 Chebyshev polynomials in z 


Defining z by (1.34), we note that if w lies on the unit circle |w| = 1 (i.e. C1), 
then (1.37) gives 
z =cosé (1.45) 


and hence, from (1.42). 
we ztV22-Ll=e. (1.46) 


Thus T,,(z) is now a Chebyshev polynomial in a real variable and so by 
our standard definition (1.1), and (1.45), (1.46), 


Tn(z) = cosnd = $(ei”? + ei”) = Sw" + Ww”). 


This leads us immediately to our general definition, for all complex z, 
namely 
T(z) = 5(w” +”) (1.47) 
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where 
Z= d(w+w7"). (1.48) 


Alternatively we may write T,,(z) directly in terms of z, using (1.42) and 
(1.43), as 


T(z) = (z+ V2 — 1)" + (2 - V2? —1)"}. (1.49) 


If z lies on the ellipse E,, the locus of (1.48) when |w| = r > 1, then it 
follows from (1.47) that we have the inequality 


$(r™ =r") <|Tr(z)| < $C" +98”). (1.50) 


In Fig. 1.7 we show the level curves of the absolute value of T5(z), and it can 
easily be seen how these approach an elliptical shape as the value increases. 


Figure 1.7: Contours of |T5(z)| in the complex plane 
We may similarly extend polynomials of the second kind. If |w| = 1, so 
that z = cos 0, we have from (1.4), 


sin n@ 


Un-1(2) = 


sin 6 © 
Hence, from (1.45) and (1.46), we deduce the general definition 


a ee a (1.51) 


w—-wt 


where again z = 3(w+w'). Alternatively, writing directly in terms of z, 


(1.52) 


If z lies on the ellipse (1.44), then it follows directly from (1.51) that 


re _ r7n re + ron 
< |Un_1(z)| < ——; 


r+r-1 — ~ pari? 


(1.53) 
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however, whereas the bounds (1.50) on |T;,,(z)| are attained on the ellipse, the 
bounds (1.53) on |U,—-1(z)| are slightly pessimistic. For a sharp upper bound, 
we may expand (1.51) into 


mt (1.54) 


& 
€ 


Un_1(z) = wrt w3 
giving us 


[Un—1(z)| < |w"*| + |w"3| Ae sae |w>-”| 4 Jui” | 


= prot pr3 ee pon pion 
n —n 
| ioe bd 

eS (1.55) 
PL 


which lies between the two bounds given in (1.53). In Fig. 1.8 we show the 
level curves of the absolute value of Us(z). 


Figure 1.8: Contours of |Us(z)| in the complex plane 


The third- and fourth-kind polynomials of degree n in z may readily be 
defined in similar fashion (compare (1.51)) by 


wets of: wror-3 
Va(z) = ——-——.— (1.56) 
w2z+w 2 
wets — wir? 
Ww2—Ww 2 


where w? is consistently defined from w. More precisely, to get round the 
ambiguities inherent in taking square roots, we may define them by 


n+1 n 


w + WwW 
wert —w” 
W,(2) = ——— (1.59) 


It is easily shown, by dividing denominators into numerators, that these give 
polynomials of degree n in z = $(w+w7"). 
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1.4.3 Shabat polynomials 


Shabat & Voevodskii (1990) introduced the concept of ‘generalised Chebyshev 
polynomials’ (or Shabat polynomials), in the context of trees and number 
theory. The most recent survey paper in this area is that of Shabat & Zvonkin 
(1994). They are defined as polynomials P(z) with complex coefficients having 
two critical values A and B such that 


Pz) =0— > Pe) =A or P(e) = B: 
The prime example of such a polynomial is T,,(z), a first-kind Chebyshev 


polynomial, for which A = —1 and B = +1 are the critical values. 


1.5 Problems for Chapter 1 


1. The equation x = cos @ defines infinitely many values of 6 corresponding 
to a given value of x in the range [—1, 1]. Show that, whichever value 
is chosen, the values of T;,(a”), Un(x), Vn(a) and W,,(x) as defined by 
(1.1), (1.4), (1.8) and (1.9) remain the same. 


2. Determine explicitly the Chebyshev polynomials of first and second 
kinds of degrees 0, 1, 2, 3, 4 appropriate to the range [—4, 6] of x. 


3. Prove that 


and that 


Um—1(Tn(@))Un—1(&) = Un—1(Tm(&))Um-1() = Umn-1(2). 


4. Verify that equations (1.33) yield the same polynomials for 7 > 1 as the 
trigonometric definitions of the Chebyshev polynomials give for |a| < 1. 


5. Using the formula 


z=4(rt+r—')cosO+ si(r—r-*)siné, (r>1) 


which defines a point on an ellipse centred at 0 with foci z = +1, 
(a) verify that 
V22-1=43(r—r—')cos6+ di(r+r7*)sind 


and hence 


(b) verify that |z +22 — 1| = 


18 
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6. By expanding by the first row and using the standard three-term recur- 


rence for T,(), show that 


-1 Qr —1 (Joes 0 0 0 
0 1 Qn —1 Pores 0 0 0 
0 0 0 0 atte —1 2x —1 
0 0 0 DO + 0 -1 & |i x nm determinant) 


Write down similar expressions for Up(x), Vn(x) and W,,(2). 


7. Given that the four kinds of Chebyshev polynomial each satisfy the same 
recurrence relation 
Xn =. 20Xpn-1 _ Xn-2; 
with Xo = 1 in each case and X, = x, 2a, 2x +1, 2x —1 for the four 
respective families, use these relations only to establish that 


(a) Vi(x) + Wi(x) = 2U;i(z), 

(b) Vi(x) — Wi(x) = 2Ui-1(2), 
(c) Uj(x) — 2T;(x) = Uj-a(2), 
(d) U;i(a) — T;(x) = cUj;_-1(z). 


8. Derive the same four formulae of Problem 7, this time using only the 
trigonometric definitions of the Chebyshev polynomials. 


9. From the last two results in Problem 7, show that 


(a) Ti(x) = cUj-1(x) — Ui-2(2), 
(b) U;(x) = 2xU;_1(x) = Uj_-2(2). 


CHAPTER 2 


Basic Properties and Formulae 


2.1 Introduction 


The aim of this chapter is to provide some elementary formulae for the manip- 
ulation of Chebyshev polynomials and to summarise the key properties which 
will be developed in the book. Areas of application will be introduced and 
discussed in the chapters devoted to them. 


2.2 Chebyshev polynomial zeros and extrema 


The Chebyshev polynomials of degree n > 0 of all four kinds have precisely n 
zeros and n+ 1 local extrema in the interval [—1, 1]. In the case n = 5, this 
is evident in Figures 1.1, 1.3 and 1.4. Note that n — 1 of these extrema are 
interior to [—1, 1], and are ‘true’ alternate maxima and minima (in the sense 
that the gradient vanishes), the other two extrema being at the end points 
+1 (where the gradient is non-zero). 


From formula (1.1), the zeros for x in [—1,1] of T,(x) must correspond to 
the zeros for 6 in [0,7] of cosné, so that 


nO=(k—s)n, (kK =1,2,...,n). 


i 
2 


Hence, the zeros of T;,(x) are 


XL = Lp, = cos ———,,_ (k= 1,2,...,n). (2.1) 


EXAMPLE 2.1: For n = 3, the zeros are 


Note that these zeros are in decreasing order in x (corresponding to in- 
creasing @), and it is sometimes preferable to list them in their natural order 
as 


ge = gs $< (RS ae yt): (2.2) 
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Note, too, that « = 0 is a zero of T,,(x) for all odd n, but not for even n, and 
that zeros are symmetrically placed in pairs on either side of 7 = 0. 


The zeros of U,,(a) (defined by (1.4)) are readily determined in a similar 
way from the zeros of sin(n + 1)6 as 


7 
XL = Yr = cos TS Cl ra |) 2.3 
m= cos, | ) (2.3) 
or in their natural order 
—k+1 
pee, eS, ahs (2.4) 
n+1 


One is naturally tempted to extend the set of points (2.3) by including the 


further values yo = 1 and yn+1 = —1, giving the set 
k 
2 = Ye = 008 Ty, (k=0,1,...,n +1). (2.5) 
These are zeros not of U,(x), but of the polynomial 
(1—2”)U,(z). (2.6) 


However, we shall see that these points are popular as nodes in applications 
to integration. 


The zeros of V,,(a) and W,,(a) (defined by (1.8), (1.9)) correspond to zeros 


of cos(n + $)6 and sin(n + 4)6, respectively. Hence, the zeros of V,,(x) occur 
at 


k-—i 
pe ee ee (2.7) 
or in their natural order 
—k+Ht 
eee oils Siw); (2.8) 
while the zeros of W,,(a) occur at 
kr 
= AAD. ces 2.9 
x = cos ae ( n) (2.9) 
or in their natural order 
—k+1 
hinge ok (k =1,2,...,n). (2.10) 


5 


Note that there are natural extensions of these point sets, by including the 
value k = n+ 1 and hence x = —1 in (2.7) and the value & = 0 and hence 
x =1 in (2.9). Thus the polynomials 


(1+2)V,(x) and (1 — 2)W,,(2) 
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have as zeros their natural sets (2.7) for k = 1,...,n+1 and (2.9) for k = 
0,1,...,m, respectively. 


The internal extrema of T,,(x) correspond to the extrema of cos n0, namely 
the zeros of sinn@, since 


d d r ae 
—T,, (x) = —cosné = sees / x 72 rnsinnd 


di dx do dé sin 
Hence, including those at « = £1, the extrema of T,,(x) on [—1, 1] are 
n= cos“, (k =0,1,...,n) (2.11) 
or in their natural order 
n= cos AO (k =0,1,...,n). (2.12) 


These are precisely the zeros of (1 — x?)U,_1(x), namely the points (2.5) 
above (with n replaced by n — 1). Note that the extrema are all of equal 
magnitude (unity) and alternate in sign at the points (2.12) between —1 and 
+1, as shown in Figure 1.1. 

The extrema of U;,(x), Vi(w), Wn(a) are not in general as readily deter- 
mined; indeed finding them involves the solution of transcendental equations. 
For example, 

d U(x) d sin(n+1)0 —(n+1)sin@cos(n+1)@+4 cos @sin(n + 1)6 
—_——- n x)= = Co 
dz sind sin? @ 
and the extrema therefore correspond to values of @ satisfying the equation 
tan(n + 1)0 = (n+1)tand 40. 


All that we can say for certain is that the extreme values of U,,(x) have 
magnitudes which increase monotonically as |x| increases away from 0, until 
the largest magnitude of n+ 1 is achieved at « = +1. 


On the other hand, from the definitions (1.4), (1.8), (1.9), we can show 
that 


V1l-—2?U,(x) = sin(n + 1)6, 
VI+2¢Vp(«) = V2c08(n + 4)6, 
VI —aW,(a) = V2sin(n + 4)0; 


Hence the extrema of the weighted polynomials V1 — x? U,(x), /1 + 2 Vn(2), 
V1—2W,,(2) are explicitly determined and occur, respectively, at 
(2k + 1)7 Qhr (2k + 1)r 


ape err oo. Ne eae a LY 1 EP 
Cee Sire ed Se cet ( at) 
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2.3. Relations between Chebyshev polynomials and powers of x 


It is useful and convenient in various applications to be able to express Cheb- 
yshev polynomials explicitly in terms of powers of x, and vice versa. Such 
formulae are simplest and easiest to derive in the case of the first kind poly- 
nomials T,,(x), and so we concentrate on these. 


2.3.1 Powers of x in terms of {T,,(x)} 


The power x” can be expressed in terms of the Chebyshev polynomials of 
degrees up to n, but, since these are alternately even and odd, we see at once 
that we need only include polynomials of alternate degrees, namely T,,(z), 
Tn—2(x), Tn—4(x), .... Writing x = cos 6, we therefore need to express cos” 6 
in terms of cosné, cos(n — 2), cos(n — 4)0, ..., and this is readily achieved 
by using the binomial theorem as follows: 


(ce 4 eH)" = pind 4 (*) eiln—2)8 og Ge i) e-i(n—2)0 4 g-ind 


= (ort ent) S$ G (he era 


+ (5) (ee ee IP lease (2.13) 


Here we have paired in brackets the first and last terms, the second and 
second-to-last terms, and so on. The number of such brackets will be 


[n/2| +1 
where |m| denotes the integer part of m. When n is even, the last bracket in 
(2.13) will contain only the one (middle) term e® [= 1]. 
Now using the fact that 
(eo? + e—#)" — (2cos 0)” = 2” cos” 6 
we deduce from (2.13) that 
[n/2] 
/ 


2”-1 eos” = S- (1) cos(n — 2k)6, 


k=0 


where the dash ( }>’) denotes that the kth term in the sum is to be halved if 
n is even and k = n/2. Hence, from the definition (1.1) of T,,(2), 


le Z2] ™ 
gg”? = gi-n > (1) Tn—2% (2), (2.14) 


k=0 
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Figure 2.1: x* (full curve) and its decomposition into Chebyshev polynomials 
(broken curves) 


where the dash now denotes that the term in To(x), if there is one, is to be 
halved. 


EXAMPLE 2.2: Taking n = 4 [see Figure 2.1]: 


8 
iN 
lI 
wo 
do 
ee 
aN 
7S 
ee 
- 
iw) 
cow 
—~ 
8 
SS 


] 
‘ 
w 


2.3.2 T,,(x) in terms of powers of x 


It is not quite as simple to derive formulae in the reverse direction. The 
obvious device to use is de Moivre’s Theorem: 


cos nO + isinné = (cos + isin @)”. 


Expanding by the binomial theorem and taking the real part, 
n 1 n-2 9 2 fe n—4 Dein 
cos n@ = cos 1— (2) cos @sin 9+ (1) cos sin” O+---. 


If sin? 6 is replaced by 1 — cos? 6 throughout, then a formula is obtained for 
cos n@ in terms of cos” 6, cos”~? 6, cos”~4 0, .... On transforming to x = cos 6, 
this leads to the required formula for T;,(x) in terms of 2”, 2"~?, 2”~4 


porte 
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We omit the details here, but refer to Rivlin (1974), where the relevant result 
is obtained in the form 


Ty(e) = 0 yee @ @ nk, (2.15) 


k=0 j=k 


However, a rather simpler formula is given, for example, by Clenshaw 
(1962) and Snyder (1966) in the form 


[n/2| 
Eis ee (2.16) 
k=0 
where 
- —k -—k-1 
cf ) = By ae 2 CG ‘ ) _ (" ‘ )| (2k < n) (2.17a) 
and 
(2h) _ (-1)* (k > 0). (2.17b) 


This formula may be proved by induction, using the three-term recurrence 
relation (1.3a), and we leave this as an exercise for the reader (Problem 5). 


In fact the term in square brackets in (2.17a) may be further simplified, 
by taking out common ratios, to give 


i n-2k-1 1 n—k 


EXAMPLE 2.3: For n = 6 we obtain from (2.17b), (2.18): 


cf) = (1°24 (2) SNe eS (18215 (5) S11. 


Te(x) = 32x° — 4807 + 182? — 1. 


Hence 


For an alternative derivation of the results in this section, making use of 
generating functions, see Chapter 5. 
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2.3.3 Ratios of coefficients in T,,(z) 


In applications, recurrence formulae which link pairs of coefficients are often 
more useful than explicit formulae (such as (2.18) above) for the coefficients 
themselves since, using such formulae, the whole sequence of coefficients may 
be assembled rather more simply and efficiently than by working them out 
one by one. 


From (2.18), 

Caaf 4 eget Ss Die ie eee eed). 

% a k 1-2---(k—1) , 
A cg ec ee 


k+1 1-2---k 
Hence, on dividing and cancelling common factors, 


(n) (n — 2k)(n — 2k — 1) (n) 
Chad = Akt Din—-k-1 * (2.19) 


where of denotes the coefficient of x” in T,,(a). Formula (2.19) is valid for 


n>Oandk> 0. 


2.4 Evaluation of Chebyshev sums, products, integrals and deriva- 
tives 


A variety of manipulations of Chebyshev polynomials and of sums or series 
of them can be required in practice. A secret to the efficient and stable exe- 
cution of these tasks is to avoid rewriting Chebyshev polynomials in terms of 
powers of x and to operate wherever possible with the Chebyshev polynomials 
themselves (Clenshaw 1955). 

2.4.1 Evaluation of a Chebyshev sum 


Suppose that we wish to evaluate the sum 
Sn = oe arP, (x) = ago Po(a) + a1 Pi (a) + +++ + GnPr(x) (2.20a) 
r=0 


where {P,-(x)} are Chebyshev polynomials of either the first, second, third or 
fourth kinds. We may write (2.20a) in vector form as 


S, =a'p, (2.20b) 
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where a? and p denote the row- and column-vectors 


a’ = (ao, @1,---, Gn), P= 


In each of the four cases, from (1.3a), (1.6a), (1.12a), (1.12b) above, the 
recurrence relation between the polynomials takes the same form 


P,(x) — 2aP,_1(a) + P,-2(x) = 0, r = 2,3,.... (2.21a) 
with Po(x) = 1 and, respectively, 
P(x) =Ti(a) =a, Pi(x) = Ui(x) = 22, 


Pi(a) = Vi(a) =2e-1, P(x) = Wi (x2) = 2a +1. (2.21b) 


Equations (2.21) may be written in matrix notation as 


1 Po(x) 1 
—22¢ 1 P,(x) XxX 
1 -2¢ 1 P (x) 0 
1 -2x¢ 1 P3(a) _| O 
1 —-2¢ 1 Pr-1(2) 0 
1 —22 1 P, (2) 0 
(2:22a) 
or (denoting the (n+ 1) x (n+ 1) matrix by A) 
Ap=c (2.22b) 
where 
1 
xX 
c=| 0 
0 
with X = —az, 0, —1, 1, respectively in the four cases. 
Let 


b? = (bo, bi, ---, bn) 
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be the row vector satisfying the equation 


1 
—2¢ 1 
1 22 1 
(bo, bi, ) bn) eee = 
1-22 1 
1 —2a 1 
= (do, G1, ---, Gn) (2.23a) 
or 
b7A =a’. (2.23b) 
Then we have 
S, =a’ p=b!’ Ap=b’ c=) +X. (2.24) 


If we write bp41 = bn42 = 0, then the matrix equation (2.23a) can be seen 
to represent the recurrence relation 


b, — 20b-41 + bpy2 = ar, r=0,1,...,n. (2.25) 


We can therefore evaluate S,, by starting with bn41 = bn+2g = O and 
performing the three-term recurrence (2.25) in the reverse direction, 


b, = 2xbp41 — bpyg tar, r=n,...,1,0, (2.26) 
to obtain b; and bo, and finally evaluating the required result S;, as 


Sn = bo + bX. (2.27) 


For the first-kind polynomials T;-(x), it is more usual to need the modified 
sum 
/ ” ‘ 1 
Si, = Yo arTp(ax) = $a0To(x) + a1 Ty (x) ++ +++ anTn (a), 
r=0 


in which the coefficient of Tp is halved, in which case (2.27) is replaced (re- 
membering that X = —x) by 


= (bo biz) t $(bo 2xb1 t bz), 


Si, = (bo — be). (2.28) 
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Note that, for a given x, carrying out the recurrence requires only O(n) 
multiplications, and hence is as efficient as Horner’s rule for evaluating a 
polynomial as a sum of powers using nested multiplication. 


In some applications, which we shall refer to later, it is necessary to eval- 
uate Chebyshev sums of a large number of terms at an equally large number 
of values of x. While the algorithm described above may certainly be used in 
such cases, one can often gain dramatically in efficiency by making use of the 
well-known fast Fourier transform, as we shall show later in Section 4.7.1. 


Sums of even only or odd only polynomials, such as 


sO = S~ G2, Tor(z) and SQ) = oe G2r41Tor41 (2) 
r=0 r=0 


may of course be evaluated by the above method, setting odd or even co- 
efficients (respectively) to zero. However, the sum may be calculated much 
more efficiently using only the given even/odd coefficients by using a modified 
algorithm (Clenshaw 1962) which is given in Problem 7 below. 


EXAMPLE 2.4: Consider the case n = 2 and x = 1 with coefficients 
ao=1, a=0.1, a2=0.001. 
Then from (2.21b) we obtain 
b3 = ba = 0 
bz = a2 = 0.01 
by = 2b2 — b3 + 0.1 = 0.12 
bo = 2b) — b2 +: 1 = 1.23. 


Hence 


2 
S~ arTr(1) = (bo — bz) = 0.61 
2 
So arUp(1) = bo = 1.23 
So arVp(1) = bo — br: = 1.11 


So arW,(1) = bo + b1 = 1.35. 
r=0 


To verify these formulae, we may set @ = 0 (i.e., = 1) in (1.1), (1.4), (1.8), 
(1.9), giving 


Tr(1) =1, Un(l) =n+4+1, (2.29a) 
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Vall) =1, Wall) =2n41. (2.29b) 


Hence 


2 
S~ a,Tr(1) = 4a +01 +02 = 0.61 


arU,(1) = ao + 2a1 + 3a2 = 1.23 


2 
So arV,(1) =aotaita2=1.11 


2 
So arW,(1) = ao + 3a; + 5a2 = 1.35. 


Incidentally, it is also useful to note that, by setting 0 = 7, in in (1.1), (1.4), 
(1.8), (1.9), we can find further special values of the Chebyshev polynomials at 


x = —1 and x = 0, similar to those (2.29) at « = 1, namely 
Trn(—1) = (-1)", Un(-1) = (-1)"(n4+ 0, (2.30a 
Vn(-D) = (-1)"(n +1), Wn(-1) =(-)", ( 


Tan4i1(0) = Uon41(0) =0, Ten(0) = U2n(0) = (—1)”, (2.30c 
—Von41(0) = Wan41(0) = (-1)", Van (0) = Won (0) = (—1)". (230d 


We leave the confirmation of formulae (2.29) and (2.30) as an exercise to the reader 
(Problem 8 below). 


2.4.2 Stability of the evaluation of a Chebyshev sum 


It is important to consider the effects of rounding errors when using recur- 
rence relations, and specifically (2.26) above, since it is known that instability 
can sometimes occur. (By instability, we mean that rounding errors grow 
unacceptably fast relative to the true solution as the calculation progresses. ) 
Three-term recurrence relations have two families of solutions, and it is possi- 
ble for contributions from a relatively larger but unwanted solution to appear 
as rounding errors; so we need to take note of this. A brief discussion is given 
by Clenshaw (1962); a more detailed discussion is given by Fox & Parker 
(1968). 


In the case of the recurrence (2.26), suppose that each b, is computed with 
a local rounding error €;, which local errors together propagate into errors 6, 
in b, for r < s, resulting in an error A in S,, or A’ in S$’. Writing 6, for the 
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computed b, and S,, or S!, for the S, or S!, computed without further error 
from (2.24) or (2.28), then from (2.26) (for fixed x) 


by = 22bp41 — bpp2 + Op — Ep (2.31) 
while 7 
b, — bp = Op. (2.32) 
Also 
Si = bo + bX, 
Si, = 3(bo — 62), 
and 


S25, = hy, S0= 8S A. 
From (2.26), (2.31), (2.32) we deduce that 
Op = 226,41 — Orga te (r<s) (2.33) 
while 


A 60 + OX, 
AP =A Gyo). 


I 


Now the recurrence (2.33), is identical in form to (2.26), with e, replacing 
ay and 6, replacing b,, while obviously dn41 = Ono = 0. Taking the final 
steps into account, we deduce that 


A=) 6F-@), (2.34) 


where P, is T,, U;, V; or W;, depending on the choice of X, and 
z / 
Ala YS eT (a). (2.35) 
r=0 


Using the well-known inequality 


a Lryr| S (= i) poeta lyr| 5 
i i i 


we deduce the error bounds 


ie / n ie / 
A’) < (>: ol max |Tp(0)| <7 ler (2.36) 


r=0 r=0 


§2.4: SUMS, PRODUCTS, INTEGRALS AND DERIVATIVES 31 


and 


Al < (Sie) nl Pe (0)1 S Cn lel (2.37) 


r=0 


where C,, = 1, n+ 1, 2n+ 1, 2n+ 1 when P, is T,, U,, V; or W,, respec- 
tively. (Note that the ¢, in these formulae are the absolute, not relative, errors 
incurred at each step of the calculation.) 


2.4.3. Evaluation of a product 


It is frequently necessary to be able to multiply Chebyshev polynomials by 
each other, as well as by factors such as x, 1 — x and 1 — 2?, and to re- 
express the result in terms of Chebyshev polynomials. Such products are 
much less readily carried out for second-, third- and fourth-kind polynomials, 
as a consequence of the denominators in their trigonometric definitions. We 
therefore emphasise T,,(x) and to a lesser extent U;,(x). 


Various formulae are readily obtained by using the substitution « = cos @ 
and trigonometric identities, as follows. 


Tmn(x)Tn (x) = cos mé cos nb = $(cos(m + n)@ + cos |m — nj 9), 


giving 
Tm(#)Tn (x) = 3(Tintn(x) + Tm—nj(2)). (2.38) 
tT,(x) = cos Ocosné = $(cos(n + 1)6 + cos |n — 1/8), 
tU,,(x) sin 8 = cos @sin(n + 1)0 = $(sin(n + 2)0 + sinnd), 
giving 
2T (2) = 2(Tn4i() + Tna\(2a)) (2.39) 
and 
tUn (2) = 3(Un41(2) + Un-1(2)), (2.40) 


(provided that we interpret U_;(x) as sin0/sin@ = 0). 


More generally, we may also obtain expressions for «™T,,(x) (and similarly 
xz'U,(x)) for any m, by expressing x” in terms of Chebyshev polynomials by 
(2.14) and then using (2.38). (See Problem 4 below.) 


In a similar vein, 


(1 —2?)T,(x) = sin? 0cosn@ = $(1 — cos 20) cos nd 
= $cosnd — F(cos(n + 2)0 + cos|n — 2| 6), 
1—27)U,(z) sin @ = sin? Osin(n + 1)0 = 4$(1 — cos 26) sin(n + 1)0 
2 
= $sin(n + 1)6 — F(sin(n + 3)0 + sin(n — 18), 
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giving 

(1 2)T,(a) =—2Tn42(@) + 8Tu(@) —2Tp-a(@) (2.41) 
and 

(1 — 2?)U,,(x) = —4Un42(x) + $Un(x) — GUn—2(2) (2.42) 
where we interpret U_,(x) as 0 again, and U_2(«) as sin(—@)/sin@ = —1. 


Note that the particular cases n = 0, n = 1 are included in the formulae 
above, so that, specifically 


tTo(x) = Ti(2), 

rUo(x) = 3Ui(2), 
(1 — 2)To(a) = $To(x) — $Ta(z), 
(1—2?)T(x) = ¢Ti(x) — ¢73(2), 
(1 — #7)Uo(a) = 3Uo(x) — 4U2(2), 
(1 —27)Ui (x) = 3Ui (a) — 7U3(z). 


2.4.4 Evaluation of an integral 


The indefinite integral of T,,(x) can be expressed in terms of Chebyshev poly- 
nomials as follows. By means of the usual substitution « = cos 6, 


[oe dz = J -eosndsind a0 


= —4 exo + 1)0— sin(n — 1)6) dé 


_ 1 {cos(n+1)@ — cos|n—1]0 
so n+1 n—1 


(where the second term in the bracket is to be omitted in the case n = 1). 


Hence 


[ tole) ae = 


Clearly this result can be used to integrate the sum 


= ss ayT,(x) 
r=0 


Se n #1; 


n=1. 


or n—-1 


[ n+1(& — Tne 1|(2) 
(2.43) 
T2( 


Ale Nle 
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in the form 


Riga / Sy, (x) de 


= 1 ar | ITp+i T,—1(2) 

= COURANT) eg +e Foon orig 
pe 

=) AE@ (2.44) 


where Ap is determined from the constant of integration, and 
A, — sae r> 0, (2.45) 
2r 


with @n41 = Gny2 = 0. 


EXAMPLE 2.5: ‘Table 2.1 gives 5-decimal values of A, computed from values of 
ar, obtained from an infinite expansion of the function e”, after each value of a, 
had been rounded to 4 decimals (numbers taken from Clenshaw (1962)). Each A, 
would be identical to a, for an exact calculation, but it is interesting to observe 


that, although there is a possible rounding error of +0.00005 in each given a,, all 
the computed A, actually have errors significantly smaller than this. 


Table 2.1: Integration of a Chebyshev series 


rT a [Ay [femora AT] 


2.93213 
1.13032 | 1.13030 0.00002 
0.27150 | 0.27150 0.00000 


0.04434 | 0.04433 0.00001 
0.00547 | 0.00548 0.00001 
0.00054 | 0.00055 0.00001 
0.00004 | 0.00004 0.00000 


There is an interesting and direct integral relationship between the Cheb- 
yshev polynomials of the first and second kinds, namely 


1 
[eo) dx = mantle) + constant (2.46) 
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(which is easily verified by substituting « = cos@). Hence, the sum 


= aU) 
r=1 


can be integrated immediately to give 


n 


fs) ) da = ss =p, (x) + constant. (2.47) 


r=1 


2.4.5 Evaluation of a derivative 


The formula for the derivative of T,,(a) in terms of first-kind polynomials is 
not quite as simple as (2.43). From (2.46) we deduce that 


Tn 41(2) =(n+1)U,(2), (2.48) 


so that it is easily expressed in terms of a second-kind polynomial. Then from 
(1.6b) and (1.7) it follows that 


dn 
ae =2n be (2.49) 


n— aa 


However, the derivative of a finite sum of first-kind Chebyshev polynomials 
is readily expressible as a sum of such polynomials, by reversing the process 
used in the integration of (2.44). Given the Chebyshev sum (of degree n + 1, 


say) 
ane 


In4i(& = 2 At) 


then 


Sp(x) = Inui = 5 arT, iB (2.50) 


where the coefficients {a,} are derived ee is given {A,.} by using (2.45) in 
the form 
Gp—-1 = Ap41 +2rA,, (r=n+1,n,...,1) (2.51a) 
with 
An+1 = An+2 = 0. (2.51b) 


Explicitly, if we prefer, we may say that 
n+1 


a=) “SL. hap (2.52) 


k=r+1 
k—r odd 
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EXAMPLE 2.6: ‘Table 2.2 shows 4-decimal values of a, computed from 4-decimal 
values of A,, for the same example as in Table 2.1. Each a, would be identical 
to A, in an exact computation, and we see this time that the derivative S;, (x) 
is less accurate than the original polynomial J,+1(a2) by nearly one decimal place. 
The contrast between these results is consistent with the principle that, in general, 
numerical integration is a stable process and numerical differentiation an unstable 
process. The size of the errors in the latter case can be attributed to the propagation, 
by (2.51a), of the error inherent in the assumptions (2.51b). 


Table 2.2: Differentiation of a Chebyshev series 


2. a5 
1.13032 
0.27150 


0.04434 
0.00547 
0.00054 


There is another relatively simple formula for the derivative of T,,(x), 
which we can obtain as follows. 


d d 
Gln) arr <cosnd)/ cos 0 
_ nsinné 
sind 
__ $n(cos(n — 1)9 — cos(n + 1)6) 
ia sin? 0 
1-2? ; 
Thus, for |z| £1, 
d atte) 


Higher derivatives may be obtained by similar formulae (see Problem 17 
for the second derivative). 


36 


2.5 


Or 


aD 
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Problems for Chapter 2 


. Determine the positions of the zeros of the Chebyshev polynomials of 


the second and third kinds for the general interval [a,b] of x. 


. From numerical values of the cosine function (from table, calculator or 


computer), determine the zeros of Ty(x), Us(x), Va(x), Wa(x) and the 
extrema of Ty(x). 


. Show that 


(a) 3U2K(x) = 3To(x) + To(x) + Ti(x) + +> + Tae (); 
(b) 4Uo¢41(x) = Ti(x) + T3(x) +--+ + Tangs (2); 
(c) U2n41(&) = To(x) + 2T2(x) +--+ + 2T24-2(x) + Tox(z). 


(Hint: In (3a), multiply by sin@ and use 2sin Acos B = sin(A — B) + 
sin(A + B). Use similar ideas in (3b), (3c).] 


. Obtain the expression 


a™T,,(x) =2-™ 2 @ Tn—m—or(2) (m<n) 


(a) by applying the formula (2.39) m times; 


(b) by applying the expression (2.14) for 7” in terms of Chebyshev 
polynomials and the expression (2.38) for products of Chebyshev 


polynomials. 
. Prove by induction on n that 
[n/2| 
T(x) = a i aia 
k=0 


(Hint: Assume the formulae are true for n = N — 2, N — 1 and hence 
derive them for n = N, using T,, = 2xT,-1 — Tn-2.] 


. Derive formulae for T* (x)T*(x) and «T*(x) in terms of {T;*(x)}, using 


the ideas of Section 2.4.3. 
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7. Suppose 


n j n 
sO) = S- AgrT2,(2), SQ) = So aar41Tar4i (2) 
r=0 r=0 


are sums of even-only/odd-only Chebyshev polynomials. 
(a) Show that s©) may be efficiently determined by applying the re- 


currence (2.26) followed by (2.28), with x replaced by (2%? —1) and 
a, replaced by a2,; 


(b) Show that gM) may be efficiently determined by applying the re- 
currence (2.26), with x replaced by (2x? — 1) and a,. replaced by 
d2r41, and then taking 


sQ) = x(bo _ bi). 
[Hint: From (1.14) and (1.15), we have T>,(x) = T;(2z? — 1) and 
Tori (x) = 2V,(2x? — 1).] 


8. Derive the formulae (2.29a)—(2.30d) for the values of T;,, Un, Vn, Wn at 
x =-1, 0, 1, using only the trigonometric definitions of the Chebyshev 
polynomials. 


9. Use the algorithm (2.21b) to evaluate 


3 


3 
eS CrT, (2), S > eUp(x), S > eV-(2), S > eW,(a) 
r=0 


r=0 


at x = —-1, 0, 1 for co = 1, c, = 0.5, co = 0.25, cz = 0.125. Check your 
results using correct values of T;., U;, V;, W, at 0, 1. 


10. Illustrate the algorithms (7a), (7b) of Problem 7 by using them to eval- 
uate at x = —1 


2 


2 2 
S~ Ca, S~ CrT2p(x), S~ CrTor41(@), 
0 0 


0 


where co = 1, cy = 0.1, c2 = 0.001. Check your results using correct 
values of T; at x = —1. 


11. Discuss the stability of the summation formulae for sums of Chebyshev 
polynomials U,., V;, W, when the size of each sum is 
(a) proportional to unity, 


(b) proportional to the largest value in [—1,1] of Un, Vn, Wn, respec- 
tively (where the sums are from r = 0 to r=n). 
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12. 


13. 


14. 


15. 


16. 
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Show that 
2(1 — 2?)\U,_2(2) = Tn(x) — Tn_2(2x); 
1 Wn—1(&) = Tn(x) + Tr-1(2); 
1—2)W,-1(@) = Tr(x) — Tr-1(2); 
) 
) 


Vin (£) Vn (x) = Tipe git) aig Trin (2) 
Win(t)Wr (2) = Tm—n|(&) — Tm4n+i (2). 


Show that Tin(x)Un—1(z) = ${Un4m—1(@) + Un—m—1(x)}, and deter- 
mine an expression for 7’U,_1(x) in terms of {U;} by a similar proce- 
dure to that of Problem 4. 
Show that (ignoring constants of integration) 

(a) [1 —22)-20, (@) de = n7¥(1 = 22) Up a (2); 

(b) [(1—«)~?V, (a) de = (n + 5)-1(1 — 2)? Wa (2); 

(c) fl +2)-?W,(2) da = (n+ 4)-1(1 + 2)? V, (2). 
Show that, for n > 0, 

<U, (2) ater a eran 
Using (2.52), show that if 
ntl 


>> ArTe(2) = Inga(2) 
r=0 


and 
n-1 
ee < ha) 
d& rl& Tepe n+1\v%), 


then 
n+1 


ary= SY) (k=r)k(k + r)Ap. 


Show further that 


Tal) = S~ (n—r)n(n + r)T,(a). 
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17. Using (2.53) and (1.3a), prove that, for n > 1, 
id? n (n+ 1)Tp—2(x) — 2nT, (x) + (n — 1)Tr42(2) 
—sT (x) = - ‘ 
da? 4 (i—22)2 
18. Show that 


19. Show that 


(1 — 2”) a cj Tj (x) = 7 (ci — ¢j42)(Tj (x) — Tj42(z)). 


CHAPTER 3 


The Minimax Property and Its Applications 


3.1 Approximation — theory and structure 


One area above all in which the Chebyshev polynomials have a pivotal role 
is the minimax approximation of functions by polynomials. It is therefore 
appropriate at the beginning of this discussion to trace the structure of the 
subject of approximation and to present some essential theoretical results, 
concentrating primarily on uniform (or £..) approximation and introducing 
the minimax property of the Chebyshev polynomials. 


It is very useful to be able to replace any given function by a simpler 
function, such as a polynomial, chosen to have values not identical with but 
very close to those of the given function, since such an ‘approximation’ may 
not only be more compact to represent and store but also more efficient to 
evaluate or otherwise manipulate. The structure of an ‘approximation prob- 
lem’ involves three central components: (i) a function class (containing the 
function to be approximated), (ii) a form (for the approximating function) 
and (iii) a norm (of the approximation error), in terms of which the problem 
may be formally posed. The expert’s job is to make appropriate selections 
of these components, then to pose the approximation problem, and finally to 
solve it. 


By a function class, we mean a restricted family F of functions f to which 
any function f(x) that we may want to fit is assumed to belong. Unless oth- 
erwise stated, we shall be concerned with real functions of a real variable, but 
the family will generally be narrower than this. For example we may consider 
amongst others the following alternative families F of functions defined on 
the real interval [a, 6]: 


1. C[a, 6]: continuous functions on [a, b}; 
2. La, b]: bounded functions on [a, b}; 
3. L5[a, b]: square-integrable functions on [a, }]; 


4. £Lp[a,b]: Lp-integrable functions on [a,b], namely functions f(x) for 
which is defined 


b 
/ w(2) [F(o)/? de, (3.1) 


where w(z) is a given non-negative weight function and 1 < p < ow. 
Note that £2[a, b] is a special case (p = 2) of L,[a, b]. 


Al 
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The reason for defining such a family of functions, when in practice we may 
only in fact be interested in one specific function, is that this helps to isolate 
those properties of the function that are relevant to the theory — moreover, 
there is a close link between the function class we work in and the norms we 
can use. In particular, in placing functions in one of the four families listed 
above, it is implicitly assumed that we neither care how the functions behave 
nor wish to approximate them outside the given interval [a, b]. 


By form of approximation we mean the specific functional form which 
is to be adopted, which will always include adjustable coefficients or other 
parameters. This defines a family A of possible approximations f*(z) to the 
given function f(a). For example, we might draw our approximation from 
one of the following families: 


1. Polynomials of degree n, with 
A=In = {f* (2) = Pn(x) =coteret+---+e,x"} (parameters {c;}) 


2. Rational functions of type (p,q), with 


ag + ay +++++ apa? 


A={ P(e) =rpale) = 1+ bj +-++++ bgx4 


\ (parameters {a;}, {b;}) 


For theoretical purposes it is usually desirable to choose the function class 
F to be a vector space (or linear space). A vector space V comprises elements 
u, UV, w, ... with the properties (which vectors in the conventional sense are 
easily shown to possess): 


1. (closure under addition) 
u+v€/Y for any u,v € Y, 


2. (closure under multiplication by a scalar) 
au € Y for any u € V and for any scalar a. 


When these elements are functions f(x), with f +g and af defined as the 
functions whose values at any point x are f(x) + g(x) and af(x), we refer 
to F as a function space. This space F typically has infinite dimension, the 
‘vector’ in question consisting of the values of f(x) at each of the continuum 
of points x in [a, }]. 


The family A of approximations is normally taken to be a subclass of F: 
AGF 


— in practice, A is usually also a vector space, and indeed a function space. 
In contrast to F, Ais a finite dimensional function space, its dimension being 
the number of parameters in the form of approximation. Thus the space II, of 
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polynomials p,,(a) of degree n has dimension n+ 1 and is in fact isomorphic 
(ie., structurally equivalent) to the space R"*! of real vectors with n + 1 
components: 

{c = (co, €1,---,n)}. 
(Note that the family of rational functions rp of type (p,q) is not a vector 
space, since the sum of two such functions is in general a rational function of 
type (p+ q, 2q), which is not a member of the same family.) 


The norm of approximation ||-|| serves to compare the function f(x) with 
the approximation f*(a), and gives a single scalar measure of the closeness of 
f* to f, namely 

Eee ie (3.2) 


Definition 3.1 A norm ||-|| is defined as any real scalar measure of elements 
of a vector space that satisfies the axioms: 


1. |lu|| > 0, with equality if and only if u= 0; 
2. \lu+v|| < ||ul| + |||] (the ‘triangle inequality’); 


3. |\|jau|| = lal |lul| for any scalar a. 


Such a definition encompasses all the key features of distance or, in the case 
of a function, size. Standard choices of norm for function spaces are the 
following: 


1. Loo norm (or uniform norm, minimax norm, or Chebyshev norm): 


fll =I flloo = max, |f(@)|:; (3-3) 


a<a<b 


2. £2 norm (or least-squares norm, or Euclidean norm): 


b 
fll = Wflle = i w(x) |f(a)|? de, (3.4) 


where w(x) is a non-negative weight function; 


3. £1 norm (or mean norm, or Manhattan norm): 


b 
fll = Ifa -| w(x) |f(@)| de; (3.5) 


4. The above three norms can be collected into the more general £, norm 
(or Holder norm): 


Sle 


b 
fll = WIFI, = / w(x) | f(x)|? w| , (1<p<oco), (3.6) 
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where w(x) is a non-negative weight function. 


With suitable restrictions on f, which are normally satisfied in practice, 
this £, norm corresponds to the £4, £2 and £; norms in the cases 
p— ~, p= 2, p= 1, respectively. 


5. The weighted minimax norm: 
Ill = max, w(x) [f(e) (3.7) 


(which does not fall into the pattern of Hélder norms) also turns out to 
be appropriate in some circumstances. 


The £, norm becomes stronger as p increases, as the following lemma 
indicates. 


Lemma 3.1 If 1< py < po < ~, and if a, b and {2 w(x) da are finite, then 
Ly, [a,b] is a subspace of Ly, [a,b], and there is a finite constant kp,p, such 
that 


IIfllp, < Ke lI Flip. (3.8) 
for every f in Ly, [a, 6). 


This lemma will be deduced from Hélder’s inequality in Chapter 5 (see 
Lemma 5.4 on page 117). 


A vector space to which a norm has been attached is termed a normed 
linear space. Hence, once a norm is chosen, the vector spaces F and A of 
functions and approximations become normed linear spaces. 


3.1.1 The approximation problem 


We defined above a family of functions or function space, F, a family of 
approximations or approximation (sub)space, A, and a measure ||f — f*|| of 
how close a given function f(z) in F is to a derived approximation f*(z) 
in A. How then do we more precisely judge the quality of f*(x), as an 
approximation to f(a) in terms of this measure? In practice there are three 
types of approximation that are commonly aimed for: 


Definition 3.2 Let F be a normed linear space, let f(x) in F be given, and 
let A be a given subspace of F. 


1. An approximation f*(x) in A is said to be good (or acceptable) if 


eer oll Sie (3.9) 


where € is a prescribed level of absolute accuracy. 


§3.1: APPROXIMATION — THEORY AND STRUCTURE 45 


2. An approximation ff(ax) in A is a best approximation if, for any other 
approximation f*(x) in A, 


lf— fell <if-f'. (3.10) 


Note that there will sometimes be more than one best approximation to 
the same function. 


3. An approximation fx(x) in A is said to be near-best within a relative 
distance p if 
If — fll < +e) If — fall, (3.11) 


where p is a specified positive scalar and f(x) is a best approximation. 


In the case of the Lo norm, we often use the terminology minimax and near- 
minimax in place of best and near-best. 


The ‘approximation problem’ is to determine an approximation of one of 
these types (good, best or near-best). In fact, it is commonly required that 
both ‘good’ and ‘best’, or both ‘good’ and ‘near-best’, should be achieved — 
after all, it cannot be very useful to obtain a best approximation if it is also 
a very poor approximation. 


In defining ‘good’ in Definition 3.2 above, an absolute error criterion is 
adopted. It is, however, also possible to adopt a relative error criterion, 
namely 

p-F 


1-Fse (3.12) 


This can be viewed as a problem of weighted approximation in which we 
require 
lw (F- fl <e (3.13) 
where, in this case, 
w(x) =1/|f(x)|. 
In approximating by polynomials on [a, b], it is always possible to obtain a 


good approximation by taking the degree high enough. This is the conclusion 
of the following well-known results. 


Theorem 3.2 (Weierstrass’s theorem) For any given f in C[a, b] and for 
any given € > 0, there exists a polynomial py, for some sufficiently large n 
such that || f — pnl|,, < €- 


Proof: A proof of this will be given later (see Corollary 5.8A on page 120). @@ 


Corollary 3.2A The same holds for ||f — pn||, for any p = 1. 
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Proof: This corollary follows directly by applying Lemma 3.1. @@ 


But of course it is a good thing from the point of view of efficiency if we 
can keep the degree of polynomial as low as possible, which we can do by 
concentrating on best or near-best approximations. 


3.2 Best and minimax approximation 


Given a norm ||-|| (such as ||-||,.., ||-||, or ||-||,), a best approximation as defined 
by (3.10) is a solution of the problem: 


inimise || f — f*|| . 3.14 
minimise | — f° (3.14) 

In the case of polynomial approximation: 
f* (x) = pr(z) = cot eae+-+++ ene", (3.15) 


to which we now restrict our attention, we may rewrite (3.14) in terms of the 
parameters as: 
minimise || f — pp]. (3.16) 
C Cn 


Oseaay 


Can we always find such a p,? Is there just one? 


Theorem 3.3 For any given p (1 < p < co), there exists a unique best 
polynomial approximation pr to any function f € Lpla,b] in the Ly norm, 
where w(x) is taken to be unity in the case p — oo. 


We refrain from giving proofs, but refer the reader to Cheney (1966), or 
other standard texts, for details. 


Note that best approximations also exist in £, norms on finite point sets, 
for 1 < p < oo, and are then unique for p 4 1 but not necessarily unique for 
p= 1. Such £, norms are defined by: 


P 


| ee dle ei |f(ai) — f*(wa)I? 


where {w;} are positive scalar weights and {2;} is a discrete set of m fitting 
points where the value of f(#;) is known. These are important in data fitting 
problems; however, this topic is away from our central discussion, and we shall 
not pursue it here. 


It is possible to define forms of approximation other than polynomials, 
for which existence or uniqueness of best approximation holds — see Cheney 
(1966) for examples. Since polynomials are the subject of this book, however, 
we shall again refrain from going into details. 
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Note that Theorem 3.3 guarantees in particular the existence of a unique 
best approximation in the £., or minimax norm. The best £.. or minimax 
approximation problem, combining (3.3) and (3.15), is (in concise notation) 


minimise max |f(x) — pp(x)|. (3.17) 
CO se Cr a<a“<b 

It is clear from (3.17) why the word ‘minimax’ is often given to this problem, 

and why the resulting best approximation is often referred to as a ‘minimax 

approximation’. 


Theorem 3.3 is not a constructive theorem and does not characterise (i.e. 
describe how to recognise) a minimax approximation. However, it is possible 
to do so rather explicitly, as the following powerful theorem asserts. 


Theorem 3.4 (Alternation theorem for polynomials) For any f(x) in 
Cla, b] a unique minimax polynomial approximation p(x) exists, and is uniquely 
characterised by the ‘alternating property’ (or ‘equioscillation property’) that 
there are n +2 points (at least) in [a,b] at which f(x) — pn(x) attains its 
maximum absolute value (namely || f — pn||,,) with alternating signs. 


This theorem, often ascribed to Chebyshev but more properly attributed 
to Borel (1905), asserts that, for p, to be the best approximation, it is both 
necessary and sufficient that the alternating property should hold, that only 
one polynomial has this property, and that there is only one best approxima- 
tion. The reader is referred to Cheney (1966), for example, for a complete 
proof. The ‘sufficient’ part of the proof is relatively straightforward and is set 
as Problem 6 below; the ‘necessary’ part of the proof is a little more tricky. 


EXAMPLE 3.1: Asan example of the alternation theorem, suppose that the function 
f(x) = 2? is approximated by the first-degree (n = 1) polynomial 


f* (x) = pi(x) = x — 0.125 (3.18) 
on [0,1]. Then the error f(x) — pn(a), namely 
x” — x + 0.125, 


has a maximum magnitude of 0.125 which it attains at x = 0, 0.5 and 1. At 
these points it takes the respective values +0.125, —0.125 and +0.125, which have 
alternating signs. (See Figure 3.1.) Hence pi(x), given by (3.18), is the unique 
minimax approximation. 


Define C$ to be the space of functions which are continuous and 27- 
periodic (so that f(27+0) = f(@)). There is a theorem similar to Theorem 3.4 
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Figure 3.1: Minimax linear approximation to x? on range [0, 1] 


which holds for approximation of a continuous function by a trigonometric 
polynomial, such as 


Gn(9) = a9 + ) (ax cos kO + by sin kO) (3.19) 
k=1 


on the range [—7, 7] of 6. 


Theorem 3.5 (Alternation theorem for trigonometric polynomials) 

For any f(@) in CS, the minimax approximation qn(@) of form (3.19) exists 
and is uniquely characterised by an alternating property at 2n +2 points of 
(—1, 7]. If bi,...,0n (or ao,...,Qn) are set to zero, so that qn(@) is a sum of 
cosine (or sine) functions alone, and if f(@) is an even (or odd) function, then 
the minimax approximation qn(0) is characterised by an alternating property 
at n+ 2 (or respectively n +1) points of [0,7]. 


Finally, we should mention recent work by Peherstorfer (1997, and else- 
where) on minimax polynomial approximation over collections of non-over- 
lapping intervals. 


3.3 The minimax property of the Chebyshev polynomials 


We already know, from our discussions of Section 2.2, that the Chebyshev 
polynomial T,,(z) has n + 1 extrema, namely 


k 
© = Y_ = cos — (k =0,1,...,7). (3.20) 
n 


Since T;,(a) = cosn@ when « = cos@ (by definition), and since cosn@ 
attains its maximum magnitude of unity with alternating signs at its extrema, 
the following property holds. 


Lemma 3.6 (Alternating property of T,,(x)) On [-1,1], Tn(x) attains 
its maximum magnitude of 1 with alternating signs at precisely (n+1) points, 
namely the points (3.20). 
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Clearly this property has the flavour of the alternation theorem for minimax 
polynomial approximation, and indeed we can invoke this theorem as follows. 
Consider the function 
f(x) = 2", 

and consider its minimax polynomial approximation of degree n—1 on [—1, 1], 
Pn—1(x), say. Then, by Theorem 3.4, f(x) — pn—1(@) = ©" — pn—i(x) must 
uniquely have the alternating property on n +1 points. But T,,(a) has a 
leading coefficient (of x”) equal to 2”~! and hence 2!~"T,,(x) is of the same 
form x” — pn—1(x) with the same alternating property. It follows that 


f= ps4 (a) = 2" "T(e): (3.21) 


We say that 2'~"T,,(x) is a monic polynomial, namely a polynomial with 
unit leading coefficient. The following two corollaries of the alternation theo- 
rem now follow. 


Corollary 3.4A (of Theorem 3.4) The minimaz polynomial approximation 
of degree n — 1 to the function f(x) =a” on [-1,1] is 


Pn—1(a) = x” — 2'-"T,, (2). (3.22) 


Corollary 3.4B (The minimax property of T,,) 24~"T;,(x) is the mini- 
max approximation on [—1,1] to the zero function by a monic polynomial of 
degree n. 


EXAMPLE 3.2: As a specific example of Corollary 3.4B, the minimax monic poly- 
nomial approximation of degree n = 4 to zero on [—1, 1] is 


2° T(x) = 279 (804 — 82? +1) = 24 — 2? +0.125. 


This polynomial has the alternating property, taking extreme values +0.125, —0.125, 
+0.125, —0.125, +0.125, respectively, at the 5 points yx = cosk7/4 (k = 0,1,...,4), 
namely 


1 
—, 0, --—=, -1l. 3.23 
Fi Fi (3.23) 
Moreover, by Corollary 3.4A, the minimax cubic polynomial approximation to the 
function f(x) = «* on [-1, 1] is, from (3.22), 


yr = 1, 


p3(x) = 2" — (a* — 2? + 0.125) = a? — 0.125, (3.24) 


the error f(x) —p3(a) having the alternating property at the points (3.23). Thus the 
minimax cubic polynomial approximation in fact reduces to a quadratic polynomial 
in this case. 


It is noteworthy that x? —0.125 is also the minimax quadratic polynomial (n = 2) 
approximation to «* on [-1,1]. The error still has 5 extrema, and so in this case the 
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alternation theorem holds with n+ 3 alternation points. It is thus certainly possible 
for the number of alternation points to exceed n+ 2. 


If the interval of approximation is changed to [0,1], then a shifted Chebyshev 
polynomial is required. Thus the minimax monic polynomial approximation of de- 
gree n to zero on (0, 1] is 


gee (i). (3.25) 


For example, for n = 2, the minimax monic quadratic is 


2-°T3 (x) = 273 (8a? — 8a +1) = 2? — 2 + 0.125. 


This is precisely the example (3.18) that was first used to illustrate Theorem 3.4 
above. 


3.3.1 Weighted Chebyshev polynomials of second, third and fourth 
kinds 


We saw above that the minimax property of T,,(x) depended on the alternating 
property of cosn@. However, an alternating property holds at n+ 1 points 0 
in [0,7] for each of the trigonometric polynomials 


k+t 
sedi: mee 22". (eg 25y 
a 
i kr 
cos(n + 5)0, at lonieraea (k =0,...,n), 
2 
k+3 
sin(n + 4)0, apg = Cran (k = 0,...,n) 
2, 


The following properties may therefore readily be deduced from the defi- 
nitions (1.4), (1.8) and (1.9) of U,(), Vn(x), Wn(x). 


Corollary 3.5A (of Theorem 3.5) (Weighted minimax properties of Un, 
Vas W,) 


The minimax approzimations to zero on [—1,1], by monic polynomials of 


degree n weighted respectively by V1 — «?, V1 +a and V1— <a, are 


2-"U, (x), 2-"V, (x) and 2-"W,, (2). 


The characteristic equioscillation may be seen in Figure 3.2. 
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Figure 3.2: Equioscillation on [—1,1] of T5(x), V1 — 2?Us(x), V1 + aV5(x) 
and V1 — «W5(a) 
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3.4 The Chebyshev semi-iterative method for linear equations 


The minimax property of the Chebyshev polynomials T;, has been exploited to 
accelerate the convergence of iterative solutions of linear algebraic equations 
(Varga 1962, p.138), (Golub & van Loan 1983, p.511). 


Let a set of linear equations be written in matrix form as 
Ax =b. (3.26) 


Then a standard method of solution is to express the square matrix A in the 
form A = M —N, where the matrix M is easily inverted (e.g., a diagonal or 
banded matrix), to select an initial vector x9, and to perform the iteration 


Mxz41 = Nx, +b. (3.27) 


This iteration will converge to the solution x of (3.26) if the spectral radius 
p(G) of the matrix G = M™!N (absolute value of its largest eigenvalue) is 
less than unity, converging at a geometric rate proportional to p(G)*. 


Now suppose that we replace each iterate x, by a linear combination of 
successive iterates: 


k 
Ya = du; (k)x; (3.28) 
where 


k 
» vj(k) = 1, (3.29) 


and write 


so that p,(1) = 1. 
From (3.26) and (3.27), we have M(x,41 — x) = N(x; — x), so that 
x; —x = GI (xo — x) 


and, substituting in (3.28) and using (3.29), 
k * 
Yr -~X= S- v5 (k)G? (xo — x) = pr(G)(xo — x), (3.30) 


where p,(G) denotes the matrix Sg v;(k)G/. 

Assume that the matrix G = M“!N has all of its eigenvalues {);} real 
and lying in the range [a, 6], where —1 < a < @ < +1. Then p;(G) has 
eigenvalues p,(A;), and 


p(p.(G)) = max [px()]| < max, [pe(A)]. (3:31) 
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Let F' denote the linear mapping of the interval [a, 3] onto the interval 
ib 1): 
2z-a-—B 


F(z) = ——_ 3.32 
() == (3.32) 
and write 5 3 
ee 
= F(1) = ——_.. 3.33 
p= FO)= (3.33) 
Choose the coefficients v;(k) so that 
Tr(F(z)) 
pr(z) = —————. (3.34) 
Tr (Ht) 
Then p,(1) = 1, as required, and 
ee pe(A)| = 1 = 1 St Den f atecorha, (3.35) 
a<A<g |T,(u)|  cosh(k argcosh j1) 


using (1.33a) here, rather than (1.1), since we know that p > 1. Convergence 
of yx to x is therefore rapid, provided that yp is large. 


It remains to show that y, can be computed much more efficiently than 
by computing x, and evaluating the entire summation (3.28) at every step. 
We can achieve this by making use of the recurrence (1.3a) in the forms 


Tr-1(H) = 2 Thu) — Tr+1(H) 


Tryi(L) = 20-7, (0) — Tr_-i(L) (3.36) 
where P a+ 
a 
pt gee gees (3.37) 


From (3.30) we have 


Yk41 — Yk-1 = (Yet 7 x) = (Yr—1 = x) 
= pr+i(G)(xo — x) — pr-1(G)(xo — x) 
= Pant 2 a) cae: 
Tee Teal 
TE) Teaf)\ 
3) a. 


Sa ake (ao Tr—1(u 
Define 


be = uo (3.38) 


Then, using (3.36), the expression 


(Ye+1 — Yk-1) — Weoil¥e — Ye-1) 
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simplifies to 


T-p 


y (Xo = %) weer (yr —X) 


= we4i7(G — 1) (ye — X) = wei 72k 
where 
y = 2/(2-—a- 8B) (3.39) 
and where z, satisfies 
Mz, = M(G —- 1)(yz — x) 
= (N-— M)(yz, — x) = A(x — yz) = b— Ayg. (3.40) 


The successive iterates yx, can thus be generated by means of the three- 
term recurrence 


Yeti = Wesilye, —Ye-1 + Ze) + ye-1, &=1,2,..., (3.41) 
starting from 
Yo=xXo0, Yi=Yot Zo, (3.42) 
where Te) 7 3 5 
kL a 
Wk = 2 ay a = a err, | ay: = ‘ey ae 
et Ta) B B-a 2-a-— 


and zx is at each step the solution of the linear system 
Mz, =b-— Ayr. (3.43) 


Using (1.3a) again, we can generate the coefficients wz most easily by means 
of the recurrence i 


=e (3.44) 


Whit = 


with w, = 2; they converge to a limit wz — 2u(p w—l)ask>o. 
In summary, the algorithm is as follows: 
Given the system of linear equations Ax = b, with A = M-N, 


where Mz = b is easily solved and all eigenvalues of M~!N lie on 
the real subinterval [a, 3] of [—1, 1]: 


2 2-—a-—6 
1. Let y := ——— and p := —— 
eee es terre B ane B-a ’ 
2. Take an arbitrary starting vector yo := Xo; 
Take wy, := 2; 


Solve Mzy = b — Ayo for z0; 
Let yi := Xo + YZo (3.42); 
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3. For k= 1,2,...: 
1 
L 4. = 44): 
et Wk4+1 1 — wp/4p? (3 5 
Solve Mz; = b — Ayy, for zz (3.43); 
Let yro1 = Weti(¥r — Ye-1 + Zk) + YK-1 (3.41). 


3.5 Telescoping procedures for power series 


If a function f(a) may be expanded in a power series which converges on 
[—1,1] (possibly after a suitable transformation of the x variable), then a 
plausible approximation may clearly be obtained by truncating this power 
series after n+ 1 terms to a polynomial p,,(x) of degree n. It may be possible, 
however, to construct an nth degree polynomial approximation better than 
this, by first truncating the series to a polynomial p,, (a) of some higher degree 
m > n (which will usually be a better approximation to f(x) than p,(a)) and 
then exploiting the properties of Chebyshev polynomials to ‘economise’ p,, (2) 
to a polynomial of degree n. 

The simplest economisation technique is based on the idea of subtracting a 


constant multiple of a Chebyshev polynomial of the same degree, the constant 
being chosen so as to reduce the degree of the polynomial. 


EXAMPLE 3.3: For f(x) = e”, the partial sum of degree 7 of the power series 
expansion is given by 


x? x ra 
p7(x) 1 x or t 3r Pecagct 7 
= 1+a+0.5a7 + 0.16666672" + 0.0416667a* + 
+ 0.0083332° + 0.0013889a° + 0.00019842", (3.45) 


where a bound on the error in approximating f(x) is given, by the mean value 
theorem, by 


e 


31 0.0000674 for x in [—1,1]. (3.46) 


8 
Fa) — pr(o)1 = |) = 


(The actual maximum error on [—1, 1] in this example is in fact the error at « = 1, 
|f(1) — pr(1)| = 0.0000279.) 


Now (3.45) may be economised by forming the degree-6 polynomial 
po(x) = pr(x) — 0.0001984 [2-°T;(a)| 
= p7(x) — 0.0000031 T; (x). (3.47) 
Since 2-°T; (x) is the minimax monic polynomial of degree 7, this means that pg¢ is 


the minimax 6th degree approximation to p7 on [—1, 1], and p7 has been economised 
in an optimal way. 
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From (3.45), (3.47) and the coefficients in Table C.2, we obtain 
po(x) = pr(x) — 0.0001984(64x" — 112x° + 56x°* — 7x) /2° 
= pr(ax) — 0.0001984(a" — 1.752° + 0.8752 — 0.1093752). 
Thus 


pe(x) = 1+ 1.00002172 + 0.52 + 0.16649312? + 
+ 0.041666727 + 0.0086805a° + 0.00138892°. (3.48) 


(Since T7(x) is an odd function of x, coefficients of even powers of x are unchanged 
from those in p7(x).) An error has been committed in replacing p7 by pe, and, from 
(3.47), this error is of magnitude 0.0000031 at most (since |T7(x)| is bounded by 1 
on the interval). Hence, from (3.46), the accumulated error in f(a) satisfies 


|f(x) — pe(a)| < 0.0000674 + 0.0000031 = 0.0000705. (3.49) 


A further economisation leads to the quintic polynomial 
ps(x) = pe(x) — 0.0013889 [2-°T5(x)] 
= pe(x) — 0.0000434 Tg (x). (3.50) 


Here ps is the minimax quintic polynomial approximation to ps. From (3.48), (3.50) 
and Table C.2, we obtain 


ps(x) = pe(x) — 0.0013889(322° — 48a? + 18a” — 1)/2° 
= pe(x) — 0.0013889(x° — 1.52 + 0.562527 — 0.03125). 
Thus 


ps(x) = 1.0000062 + 1.0000217a + 0.4992188a7 + 
+ 0.16649312° + 0.0437500x* + 0.0086805a:° (3.51) 


and, since T(x) is an even function of x, coefficients of odd powers are unchanged 
from those in pe(x). The error in replacing pe by ps is, from (3.50), at most 
0.0000434. Hence, from (3.49), the accumulated error in f(a) now satisfies 


|f (x) — ps(x)| < 0.0000705 + 0.0000434 = 0.0001139. (3.52) 


Thus the degradation in replacing p7 (3.45) by ps (3.51) is only marginal, increasing 
the error bound from 0.000067 to 0.000114. 


In contrast, the partial sum of degree 5 of the power series (3.45) has a mean- 
value-theorem error bound of |x°e°/6!| < e/6! ~ 0.0038 on [-1, 1], and the actual 
maximum error on [—1, 1], attained at x = 1, is 0.0016. However, even this is about 
15 times as large as (3.52), so that the telescoping procedure based on Chebyshev 
polynomials is seen to give a greatly superior approximation. 


The approximation (3.50) and the 5th degree partial sum of the Taylor series are 
both too close to e” for the error to be conveniently shown graphically. However, in 
Figures 3.3 and 3.4 we show the corresponding approximations of degree 2, where 
the improved accuracy is clearly visible. 
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ser 


Figure 3.3: The function e” on Figure 3.4: The function e” on 
[—1,1] and an economised polyno- [—1,1] and its Taylor series trun- 
mial approximation of degree 2 cated at the 2nd degree term 


An alternative technique which might occur to the reader is to rewrite the poly- 
nomial p7(a), given by (3.45), as a sum of Chebyshev polynomials 


7 
p7(x“) = SS crTh (2), (3.53) 
k=0 
where c, are determined by using, for example, the algorithm of Section 2.3.1 above 
(powers of x in terms of {T;,(x)}). Suitable higher order terms, such as those in T6¢ 
and T7, could then be left out of (3.53) according to the size of their coefficients cx. 
However, the telescoping procedure above is exactly equivalent to this, and is in fact 
a somewhat simpler way of carrying it out. Indeed c7 and cg have been calculated 
above, in (3.47) and (3.50) respectively, as 


c7 = 0.0000031, ce = 0.0000434. 


If the telescoping procedure is continued until a constant approximation po(x) is 
obtained, then all of the Chebyshev polynomial coefficients c, will be determined. 


3.5.1 Shifted Chebyshev polynomials on [0,1] 


The telescoping procedure may be adapted to ranges other than [—1, 1], pro- 
vided that the Chebyshev polynomials are adjusted to the range required. 
For example, the range [—c,c] involves the use of the polynomials T;,(a/c). 
A range that is often useful is [0,1] (or, by scaling, [0,c]), and in that case 
the shifted Chebyshev polynomials Tf (a) (or Ty(a/c)) are used. Since the 
latter polynomials are neither even nor odd, every surviving coefficient in the 
polynomial approximation changes at each economisation step. 
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EXAMPLE 3.4: Suppose that we wish to economise on [0, 1] a quartic approximation 


to f(x) =e”: 
qa(z) = 1+ a4 0.527 + 0.1666667x° + 0.0416667x* 


in which the error satisfies 


f(x) — qa(x)| = =e < = = 0.0227. (3.54) 
Then the first economisation step leads to 
q3(x) = qa(x) — 0.0416667 [2-’ Ty (x)] 
= qa(x) — 0.0003255 Ty (x). (3.55) 


From Table C.2: 


q3() = qa(a) — 0.0416667(128a* — 256a° + 1602? — 32a + 1)/27 
= qa(x) — 0.0416667(2* — 2x? + 1.252? — 0.25a + 0.0078125). 
Thus 
qa(v) = 0.9996745 + 1.01041662 + 0.44791672 + 0.252". (3.56) 


Here the maximum additional error due to the economisation is 0.0003255, from 
(3.55), which is virtually negligible compared with the existing error (3.54) of qa. In 
fact, the maximum error of (3.56) on [0, 1] is 0.0103, whereas the maximum error of 
the power series truncated after the term in z° is 0.0516. 


The economisation can be continued in a similar way for as many steps as 
are valid and necessary. It is clear that significantly smaller errors are incurred 
on [0,1] by using T (a) than are incurred on [—1, 1] using T;,(z). This is to 
be expected, since the range is smaller. Indeed there is always a reduction in 
error by a factor of 2”, in economising a polynomial of degree m, since the 
respective monic polynomials that are adopted are 


Bomgy Ce) and. ose) 


3.5.2 Implementation of efficient algorithms 


The telescoping procedures above, based on T;,(x) and T(x) respectively, are 
more efficiently carried out in practice by implicitly including the computation 
of the coefficients of the powers of x in T;, or T; within the procedure (so that 
Table C.2 does not need to be stored). This is best achieved by using ratios 
of consecutive coefficients from formula (2.19) of Section 2.3.3 above. 
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Consider first the use of the shifted polynomial T;(#/d) on a chosen range 
(0, d]. Suppose that f(a) is initially approximated by a polynomial p,,(x) of 
degree m, where for each  < m, 


= Soak Ope S104 Ons cou OP (3.57) 


Then the first step of the telescoping procedure replaces p,, by 
Pm—1(£) = Pm(x) — a™2!-2™g™T* (x /d). (3.58) 


(The factor d™ is included, to ensure that 2'~?"d™T* (a/d) is monic.) 


Now, write 
Paige a aka" (3.59) 


where g2m—1 gin) is the coefficient of x* in T*,(x). Then, by (3.57), (3.58), 
(3.59): 


ab?) = a™ —a™d™ = (k=m—1,m—2,...,0). (3.60) 


The index k has been ordered from k = m — 1 to k = 0 in (3.60), since the 
coefficients a”) will be calculated in reverse order below. 


Now T(x) = T2m(x2) and hence, from (2.16), 
ae i as (3.61) 


where cen) is defined by (2.17a). Hence, in (3.59), 
a) =O) oem, (3.62) 
Now, from (2.19) 


(n) (n — 2k)(n — 2k — 1) of!) 


= OO 3.63 
“e+ ~ 4k + 1)(n—k—1) oe) 

and hence, from (3.62), 

a™ — Orn — Fem = AED gion 
m—k-1 ~ " A(k+ 1)(Qm—k—1) ™* 
Thus 
—2r(2r—1 

a” = —er(2r- 1) em (3.64) 


A(m—r+1)(m+r—1) 
where qd =1 


In summary, the algorithm is as follows: 
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Given pm(x) of form (3.57), with coefficients al”: 


1. With d” =1, determine d&”,,...,d°”, using (3.64); 


2. Determine qe using (3.60), and hence pm—1(x) of form 


(3.57), with coefficients a 


However, if a telescoping procedure is based on the range [—d, d] and the 
standard polynomials T;,(x/d), then it is more appropriate to treat even and 
odd powers of x separately, since each Tj, involves only one or the other, and 
so the algorithm is correspondingly more complicated, but at the same time 
more efficient. 


Suppose f(x) is initially approximated by the polynomial p2j7+1(x) of odd 
degree, where (for each ¢ < M) 


e e 
pe+1(£) = S- pt) 2+ + S- cl)? (3.65a) 
k=0 k=0 
and 
é-1 l 
poe(x) = S- Be gett + Se Og? (3.65b) 
k=0 k=0 


Then the first two (odd and even) steps of the telescoping procedure replace 
pem41(%) by pom (x) and pom (x) by pew—1(“), where 


pom (®) = ponrga(a) — bYy?2--4 PMH Tor yi(a/d), — (3.66a) 
pom—1() = pom(a) — cH 21-2M PMT v¢(a2/d). (3.66b) 


Now, let 22M (M) and pom) denote respectively the coefficients of 2?*+1 
in T2741(x) and of x?* in Toyy(z). 


Then, from (2.16), 


M 
Tom4i(2/d) = 2?@q-2M-1 s- e(M) p2k+1 g2M—2k _ 


k=0 


ee (3.67a) 


Me 


> 
ll 
° 


M 
Toy (e/d) = eM a Sa a 
k=0 


cM) 2k gk (3.67b) 


Me 


> 
Il 
° 
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Hence, from (3.65)—(3.67), 


BM) — ol) _ G9) (k= M—1,M—2,...,0),  (3.68a) 


M-1) _ ol) 


cf — cM) 7M) (k= M-1,M—2,...,0).  (3.68b) 


Formulae for generating the scaled Chebyshev coefficients eM) and ‘ae 


may be determined from (3.63) and (3.67) (by replacing n by 2M + 1, 2M, 
respectively) in the form 


fib, = 2M =2k-+ 2M = 26) an 
M—k-1 A(k + 1)(2M — k) M-—k? 
Gh. 2 eM 2M 2k) 
M-k-1 Ak+)@QM—k-1) ¢M-* 
Thus el) = ia = 1, and 
mm) RAr+YQ2r) oan 
eri "40 —rt fen ’ (3.69a) 
ay NG) gan 
ae 4(M—r+l)(M+r— pir (3.69b) 


In summary, the algorithm is as follows: 


Given pov+i(x) of form (3.65a), with coefficients p™) and 


o: 
1. With el) = 1, determine ee ae eM) using (3.69a); 
2. Determine pM), using (3.68a), and hence poy(x) of form 


(3.65b), with coefficients gu) and cM), 
3. With f( =1, determine f,,..., fC, using (3.69b); 


4. Determine (Mo) using (3.68b), and hence poas_1(x) of form 


(3.65a), with coefficients piM@—1) and ees 


We should add as a postscript that Gutknecht & Trefethen (1982) have 
succeeded in implementing an alternative economisation method due to Cara- 
théodory and Fejér, which yields a Chebyshev sum giving a much closer ap- 
proximation to the original polynomial. 
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3.6 The tau method for series and rational functions 


Sometimes a power series converges very slowly at a point of interest, or even 
diverges, so that we cannot find a suitable partial sum to provide an initial 
approximation for the above telescoping procedure. However, in some cases 
other approaches are useful, one of which is the ‘tau’ (7) method! of Lanczos 
(1957). 


Consider for example the function 


which has the power series expansion 


l-gta? oF eee + (—1) 2" +45 - 


This series has radius of convergence 1, and since it does not converge for 
|x| > 1, cannot be used on [0, 1] or wider ranges. However, y() is the solution 
of the functional equation 

(14+ 2)y(z) =1 (3.70) 


and may be approximated on [0,1] by a polynomial p,(x) of degree n in the 
form 


pn(x) = S~ ex Tf (a) (3.71) 
k=0 


(where, as previously, the dash denotes that the first term in the sum is 
halved), by choosing the coefficients {c,} so that p, approximately satisfies 
the equation 

(1+ 2)pn(x) = 1. (3.72) 


Equation (3.72) can be perturbed slightly into one that can be satisfied 
exactly, by adding to the right-hand side an undetermined multiple 7 (say) of 
a shifted Chebyshev polynomial of degree n + 1: 


(+ a)p,(@) = 14 Tf (2). (3.73) 


Since there are n + 2 free parameters in (3.71) and (3.73), namely cy (k = 
0,1,...,) and 7, it should be possible to determine them by equating coeffi- 
cients of powers of x in (3.73) (since there are n+2 coefficients in a polynomial 
of degree n +1). Equivalently, we may equate coefficients of Chebyshev poly- 
nomials after writing the two sides of (3.73) as Chebyshev summations; this 
can be done if we note from (2.39) that 


(Qa — 1)T, (2a — 1) = $[Th41(2e — 1) + T,-1) (22 — 1)] 


1A slightly different but related approach, also known as the ‘tau method’, is applied to 
solve differential equations in a later chapter (see Chapter 10). 
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and hence, since T(x) = T,(2x — 1), 
(1+ a) Tp (x) = ze (e) + OTE (x) + Tes (@)]- (3.74) 
Substituting (3.74) into (3.71) and (3.73), 


n 


Sy deg Ta (a) + 6TH (x) + Ty(2)] = Te (a) + 774 (2). 


k=0 
On equating coefficients of Tj, ..., 77,1, we obtain 
+(3c9 +.e1) = 1, 
t(Ce-1 + 6cK + Ck41) = 0 (k =1,...,.n- 1), 

t(Cn—1 +6cn) = 0, 

he ta 
These are n + 2 equations for co, C1, ..-, C, and 7, which may be readily 
solved by back-substituting for c,, in terms of 7, hence (working backwards) 
determining Cp_1 , Cn—2, ---,; Co in terms of 7, leaving the first equation to 


determine the value of T. 


EXAMPLE 3.5: For n = 3, we obtain (in this order) 

c3 = Ar, 

c2 = —6c3 = —247, 

cy = —6c2 —c3 = 1407, 

co = —6c1 — co = —8167, 

3co +¢1 = —23087 = 4. 
Hence 7 = —1/577 and, from (3.71), 
y(z) = ps(x) 
= 577 [40875 (x) — 14077 (x) + 24T3 (x) — 4T3 (x)| 

= 0.707106T9 (x) — 0.242634TS (a) + 


+ 0.041594TS (x) — 0.006932T; (2). (3.75) 
The error e(x) in (3.75) is known from (3.70) and (3.73) to be 
TT; (a) 


e(a) = yl) = pala) = FAS), 


Since 1/(1 + x) and T(x) are both bounded by 1 in magnitude, we deduce the 
bound 

je(x)| < |r| = 0.001704 ~ 0.002 on (0, 1]. (3.76) 
This upper bound is attained at x = 0, and we would expect the resulting approxi- 
mation p3(x) to be reasonably close to a minimax approximation. 
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3.6.1 The extended tau method 


Essentially the same approach has been proposed by Fox & Parker (1968) for 
the approximation on [—1, 1] of a rational function a(«) /b(x) of degrees (p, q). 
They introduce a perturbation polynomial 


n+q 


e(x) = Ne Finn Lin (@) (3.77) 


m=n+1 


in place of the single term TT;,41(x) used above, to give 


a(x) + e(x) = d(x) S~ Chl p(x). (3.78) 
k=0 


The number and degrees of the terms in (3.77) are chosen so that (3.78) is 
uniquely solvable for {c,} and {Tm}. 


For example, for 
ae) 1l-ae+2% 


b(z)  1l+a4a? 


we need two tau terms and (3.78) becomes 


n+2 n ; 
(l-at2?)+ So tn—nTm(x) =(1+a+2°) S> ceTe(x). (3.79) 
m=nt+1 k=0 


Both sides of (3.79) are then written in terms of Chebyshev polynomials, and 
on equating coefficients, a set of equations is obtained for cy, and T,. Back- 
substitution in terms of 7; and 7» leads to a pair of simultaneous equations 
for 7, and 7; hence cz are found. 


EXAMPLE 3.6: For n = 2, (3.79) becomes, using (2.38) to transform products into 
sums, 


($To(a) — Ti(a) + $T2(x)) + 1T3(x) + r2Ta(z) 
= ($To(x) + Ti(x) + $T2(x))($coTo(x) + e1Ti (x) + c2T2(a)) 
= (3c0 + Ser + Fe2)To(x) + (Feo + Fer)Ti (x) + 
+ (4c0 + $e1 + 3c2)T2(x) + (4e1 + $c2)T3(x) + 474 (zx). 


Equating coefficients of the Chebyshev polynomials To(x), ..., T4(a) yields the 
equations 
c2 = AT. 
Ci ar 2c2 = Ary (3.80) 
Co +2c1 +6c2 = 2 


2c0 Tr Tc. = —4 
3c9 + 2c1 +2 = 6 


(3.81) 
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Back-substituting in (3.80): 
c2 = 4tT2, C1 = 471 — 872, Co = 2-— 871 — BID. 
Now (3.81) gives 
371 — 1672 = —2 


971 +472 = 0 
and hence 
7 = —18/91, 72 = 8/91 
so that 
co = 262/91, c, = —136/91, co = 32/91. 
Thus 


65 


y(x) = SS = WN s(x) = 1.439 To(x) — 1.494 Ti (x) + 0.352 To(x) 


and the error is given by 

_ 71T3(x) + 72T4(x) 
l+a+42? 

_ 0.198 T3(x) — 0.088 Ta (a) 

7 l+a2+2? ; 


e(x) = y(a) — pa(w) = 


On [-1, 1], 1/(1+ 2+ 2?) is bounded by and |T3| and |T4| are bounded by 1. 


Hence we have the bound (which is not far from the actual maximum error) 


|e(x)| < 1.333 (0.198 + 0.088) = 0.381. 


With an error bound of 0.381, the approximation found in this example is not 


particularly accurate, and indeed a much higher degree of polynomial is needed 


to represent such a rational function at all reasonably, but the method does give 


credible and measurable results even in this simple case (see Figure 3.5). 


We may note that an alternative approach to the whole calculation is to 


use the power form for the polynomial approximation 


n 
Dn(x) = S- apa® 
k=0 


and then to replace (3.79) by 


n+2 n 
(l-at+a?)+ So tin—nTm(2) = (1 +042") S- aga®. 
m=nt+1 k=0 


We then equate coefficients of powers of x and solve for 7, and 7». 


(3.82) 


(3.83) 
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i 


Figure 3.5: Rational function and a quadratic approximation obtained by the 
extended 7 method 


EXAMPLE 3.7: For n = 2, equation (3.83) takes the form 


(1— a+?) +71 (4a° — 3x) + 72(8a* — 827 +1) = (14+ 44 27)(a0 + aia + aa’), 


4 


and on equating coefficients of 1, 7, ..., 2°, 
aon = 1+72 
84 
aj+ag = —-1-3n \ (3.84) 
azt+ait+ao = 1-87 
a2+a1 = 4 (3.85) 
a2 = 8T2 


Back-substituting in (3.85): 
a2 = 872, a1 = 471 — 872, ag = 1 — 471 — 872. 
Now (3.84) gives 
47, + 972 = 0 
37, — 1672 = —2 


and hence 
T = —18/91, 72 = 8/91 
(the same values as before) so that 
ao = 99/91, a1 = —136/91, a2 = 64/91. 


Thus 
2 


8) LSet oy = : 
vo) = Fay = Type pat ~ Pale) = 1088 — 1.4952 + 0.7032”. 


It is easily verified that this is precisely the same approximation as was obtained 


previously, but expressed explicitly as a sum of powers of x. 
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For the degrees n of polynomial likely to be required in practice, it is not 


advisable to use the power representation (3.82), even though the algebra ap- 
pears simpler, since the coefficients ax, tend to become large as n increases, 
whereas the Chebyshev coefficients cz, in the form (3.71) typically tend to con- 
verge with n to the true coefficients of an infinite Chebyshev series expansion 
(see Chapter 4). 


3.7 


Problems for Chapter 3 


. Verify the axioms of a vector space for the following families of functions 


or data: 

(a) F =C[a, bj; 

(b) F = {{f(az), k = 1,...,m}} (values of a function at discrete 
points). 


What are the dimensions of these spaces? 


. Verify, from the definition of a norm, that the following is a norm: 


b P 
fll = Wfll, = / | F(«)|? | (1 <p<ov), 


by assuming Minkowski’s continuous inequality: 


(fir+a? an)’ <(firr ar)" +( flor ar)”. 


Prove the latter inequality for p = 1, 2, and show, for p = 2, that equality 
does not occur unless f(x) = Ag(x) (‘almost everywhere’), where is 
some constant. 


. For what values of p does the function f(a) = (1 — x?)~!/? belong to 


the function space £,[—1, 1], and what is its norm? 


. Prove Minkowski’s discrete inequality: 


(Zmene)' <(Lmr)'+(Ewr) 


in the case p = 2 by first showing that 


(© uve) < SG): So (ve). 
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Deduce that 


SIR 


IIfll, = bs stew (1 < p< oo) 
k=1 


is a norm for space (b) of Problem 1. 


Find proofs in the literature (Hardy et al. 1952, for example) of both 
continuous and discrete Minkowski inequalities for general p. Can equal- 
ity occur for p= 1? 


. Find the minimax constant (i.e., polynomial of degree zero) approxi- 


mation to e® on [—1,1], by assuming that its error has the alternating 
property at —1, +1. Deduce that the minimax error in this case is sinh 1. 


Generalise the above approach to determine a minimax constant ap- 
proximation to any monotonic continuous function f(z). 


. Prove the sufficiency of the characterisation of the error in Theorem 3.4, 


namely that, for a polynomial approximation p,, of degree n to a contin- 
uous f to be minimax, it is sufficient that it should have the alternating 
property at n+ 2 points 71 < +--+ < &p4o. 

(Hint: Assume that an approximation p/, exists with smaller error norm 
than p,, show that p,, — p}, changes sign between each pair x; and xj41, 
and hence obtain the result.] 


. Consider the function 


F(a) = So eTy(2), (x) 


1=0 


where {c;} are so defined that the series is uniformly convergent and 
where 6 is an odd integer not less than 2. Show that, for every 7 > n with 
n fixed, T,: has the alternating property on a set of b” + 1 consecutive 
points of [—1, 1]. Deduce that the partial sum of degree b” of (*) (namely 
the sum from i = 0 to n) is the minimax polynomial approximation of 
degree b” to f(z). 

[Note: A series in {7;,(x)} such as (*) in which terms occur progressively 
more rarely (in this case for k = 0, b, b?, b°, ...) is called lacunary; see 
Section 5.9 below for a fuller discussion. ] 


. For f(x) = arctan z, show that (1 + 2?) f’(x) = 1, and hence that 


(1422) f(x) +20(n—1) f-) (a) +(nt1)(n+2) f(a) =0 (n> 2). 


Deduce the Taylor—Maclaurin expansion 


3 ge? gt gentl 


(+) 


§3.7: PROBLEMS FOR CHAPTER 3 69 


10. 


11. 


12. 


13. 


Estimate the error in the partial sum P;(x) of degree 7 of (**) for x in 
{[—0.3, 0.3]. 

Telescope P;, into polynomials P; of degree 5 and P3 of degree 3 by 
using Chebyshev polynomials normalised to [—0.3,0.3], and estimate 
the accumulated errors in Ps and P3. 


. Given 


w 


x Oe, 
pee en sh ee ees 
3 Ck ee, SH) 


rm] 8, 


f(a) =log(1 +2) =2 


use the mean value theorem to give a bound on the error on [0, 0.1] of the 
partial sum P,, of degree n of (***). Telescope P, into polynomials P3 of 
degree 3 and Py» of degree 2, respectively, using a Chebyshev polynomial 
adjusted to [0,0.1], and estimate the accumulated errors in each case. 


(Programming Exercise) Write a computer program (in a programming 
language of your own choice) to implement the telescoping algorithm of 
Section 3.5, either 

(a) based on T;‘(x/d) and using (3.60)—(3.64) or 

(b) based on T;,,(a/d) and using (3.68)—(3.69). 


Apply the tau method of Section 3.6 to determine a polynomial approx- 
imation of degree 3 to 7/(1+ x) on [0,1] based on the equation 


(l+z)y=a 
and determine a bound on the resulting error. 


Apply the extended tau method of Section 3.6.1 to determine a polyno- 
mial approximation of degree 2 to (1+a+27)~! on [-1, 1] and determine 
a bound on the resulting error. 


Show that 27" V1 — x#?U, (x), 2-"/14+ a2V,(z) and 27"V1— «W,,(z) 
equioscillate on (n + 2), (n +1) and (n + 1) points, respectively, of 
[—1, 1], and find the positions of their extrema. Deduce that these are 
minimax approximations to zero by monic polynomials of degree n with 
respective weight functions /1—2?, /1+ 2, /1—2. Why are there 
more equioscillation points in the first case? 


CHAPTER 4 


Orthogonality and Least-Squares Approximation 


4.1 Introduction — from minimax to least squares 


The Chebyshev polynomials have been shown in Chapter 3 to be unique 
among all polynomials in possessing a minimax property (Corollaries 3.4B, 
3.5A), earning them a central role in the study of uniform (or £..) approxima- 
tion. This property is remarkable enough, but the four families of Chebyshev 
polynomials have a second and equally important property, in that each is 
a family of orthogonal polynomials. Thus, the Chebyshev polynomials have 
an important role in £2 or least-squares approximation, too. This link with 
£2 approximation is important in itself but, in addition, it enables ideas of 
orthogonality to be exploited in such areas as Chebyshev series expansions 
and Galerkin methods for differential equations. 


Orthogonal polynomials have a great variety and wealth of properties, 
many of which are noted in this chapter. Indeed, some of these properties 
take a very concise form in the case of the Chebyshev polynomials, making 
Chebyshev polynomials of leading importance among orthogonal polynomi- 
als — second perhaps to Legendre polynomials (which have a unit weight 
function), but having the advantage over the Legendre polynomials that the 
locations of their zeros are known analytically. Moreover, along with the Leg- 
endre polynomials, the Chebyshev polynomials belong to an exclusive band 
of orthogonal polynomials, known as Jacobi polynomials, which correspond 
to weight functions of the form (1 — x)°(1 +2) and which are solutions of 
Sturm—Liouville equations. 


The Chebyshev polynomials have further properties, which are peculiar 
to them and have a trigonometric origin, namely various kinds of discrete 
orthogonality over the zeros of Chebyshev polynomials of higher degree. In 
consequence, interpolation at Chebyshev zeros can be achieved exceptionally 
inexpensively (Chapter 6) and Gauss quadrature methods based on Cheby- 
shev zeros are extremely convenient (Chapter 8). 


The continuous and discrete orthogonality of the Chebyshev polynomials 
may be viewed as a direct consequence of the orthogonality of sine and cosine 
functions of multiple angles, a central feature in the study of Fourier series. It 
is likely, therefore, that a great deal may be learned about Chebyshev series 
by studying their links with Fourier series (or, in the complex plane, Laurent 
series); this is considered in Chapter 5. 


Finally, the Chebyshev polynomials are orthogonal not only as polynomi- 
als in the real variable x on the real interval [—1, 1] but also as polynomials in 
a complex variable z on elliptical contours and domains of the complex plane 
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(the foci of the ellipses being at —1 and +1). This property is exploited in 
fields such as crack problems in fracture mechanics (Gladwell & England 1977) 
and two-dimensional aerodynamics (Fromme & Golberg 1979, Fromme & 
Golberg 1981), which rely on complex-variable techniques. More generally, 
however, many real functions may be extended into analytic functions, and 
Chebyshev polynomials are remarkably robust in approximating on [—1, 1] 
functions which have complex poles close to that interval. This is a conse- 
quence of the fact that the interval [—1, 1] may be enclosed in an arbitrarily 
thin ellipse which excludes nearby singularities. 


4.2 Orthogonality of Chebyshev polynomials 


4.2.1 Orthogonal polynomials and weight functions 
Definition 4.1 Two functions f(x) and g(x) in Le[a, b] are said to be orthog- 


onal on the interval [a,b] with respect to a given continuous and non-negative 
weight function w(x) if 


b 
/ w(x) f(a)g(x) dx = 0. (4.1) 
If, for convenience, we use the ‘inner product’ notation 


b 
Gye / w(2) f(a)9(x) de, (4.2) 


where w, f and g are functions of x on [a, b], then the orthogonality condition 
(4.1) is equivalent to saying that f is orthogonal to g if 


(f, 9) =0. (4.3) 


The formal definition of an inner product (in the context of real functions 
of a real variable — see Definition 4.3 for the complex case) is as follows: 


Definition 4.2 An inner product (-, -) is a bilinear function of elements 
f,g,h,... of a vector space that satisfies the axioms: 


1. (f , f) > 0 with equality if and only if f = 0; 
2. (f,9)={9, f); 
3. (f+g,h)=(f,h) + (9g, h); 


4. (af, 9g) =a(f, g) for any scalar a. 
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An inner product defines an £L2-type norm 
fll = llflla = VF, f). (4.4) 


We shall adopt the inner product (4.2) (with various weight functions) and 
the associated £2 norm (4.4), which is identical to that defined in Chapter 3 
(3.4), through most of the remainder of this chapter. 


Here we shall in particular be concerned with families of orthogonal poly- 
nomials {¢;(x), i = 0,1,2,...} where ¢; is of degree i exactly, defined so 
that 


($i, $3) =0 GF J). (4.5) 
Clearly, since w(x) is non-negative, 
(Gi, bi) = |9il|? > 0. (4.6) 


The requirement that ¢; should be of exact degree 7, together with the or- 
thogonality condition (4.5), defines each polynomial ¢; uniquely apart from a 
multiplicative constant (see Problem 3). The definition may be made unique 
by fixing the value of (¢;, ¢;) or of its square root ||¢;||. In particular, we say 
that the family is orthonormal if, in addition to (4.5), the functions {¢;(a)} 
satisfy 

||¢4|| = 1 for all 7. (4.7) 


4.2.2 Chebyshev polynomials as orthogonal polynomials 
If we define the inner product (4.2) using the interval and weight function 
[a,b] = [t,t w(x) = (1—2)-2, (4.8) 


then we find that the first kind Chebyshev polynomials satisfy 


= cosi@ cos j8d0 (4.9) 
0 


(shown by setting « = cos@ and using the relations T;(a) = cosi@ and dx = 
—sin6 dé = —V1— 2? d8). 


Now, for 7 # J, 
| cosié cos j0d@ = i/ [cos(i + 7)@ + cos(i — 7)6] dé 
0 0 


1 “4 + 7)0 " sin(i — j)0]” a. 


1 
Zo. Sta tang, «lhe 
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Hence 
(T;,T))=0 (t# 43), (4.10) 


and {T;(x), 7 = 0,1,...} forms an orthogonal polynomial system on [—1, 1] 


with respect to the weight (1 — x2)~2. 
The norm of T; is given by 


IZ? = (Li, Ti) 


— i (cos 10)? dé 
0 


= in, (4.11a) 


while 
IIZoll? = (To, To) = (1, 1) = 7. (4.11b) 


The system {T;} is therefore not orthonormal. We could, if we wished, scale 
the polynomials to derive the orthonormal system 


V1/r To(x), { 2/n T;(x), i= 1,2,.. . 


but the resulting irrational coefficients usually make this inconvenient. It is 
simpler in practice to adopt the {T;} we defined initially, taking note of the 
values of their norms (4.11). 


The second, third and fourth kind Chebyshev polynomials are also orthog- 
onal systems on [—1, 1], with respect to appropriate weight functions: 


e U;(x) are orthogonal with respect to w(x) = (1 — ax2)?; 


e V;(x) are orthogonal with respect to w(x) = (1+ 2)2(1—2)7?; 


Nie 


e W;,(z) are orthogonal with respect to w(x) = (1+ 2)~2(1—2) 


These results are obtained from trigonometric relations as follows (using the 
appropriate definition of (-, -) in each case): 


Wi, Uy) = f 1 2%)4 U(@)U;(a) de 


= / (1 — a?)~? (1 — 2”)2U;(zx) (1 — x?)?U; (x) dx 


1 
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a | sin(¢ + 1)@ sin(j + 1)@d6 
0 
(since sin 6 U;(x) = sin(i + 1)6) 
=5 | [cos(i — j)@ — cos(i + j + 2)6| dé 
0 


0 (#9). 


I 


ai (1 — 22)" (1 +2)2V;(2) (1 +.2)?V;(2) de 


=2} cos(i + $)0 cos(j + $)0d0 
0 


(since (1 + x)? = (1 +cos6)? = (2cos? 46)? = V2cos 46 
and (1+ x)?V;(x) = V2cos(i + 4)4) 
- | [cos(¢ + 7 + 1)@ + cos(i — 7) 6] dé 
0 


=0 (#3). 


i) 


Ww) = ff (is) da? W@W) a 
=/ (1 — 22)-2 (1 — 2)? W,(2) (1 — 2)? W;(2) de 


z 2 | sin(i + 1)6 sin(j + 1)0 40 
(since (1 — x)? = (1 — cos0)? = (2sin? 46)? = V2sin 0 
and (1 — x)?W;(x) = V2sin(i + 4)@) 
é [ost j)8 — cos + j + 106] 49 


=0 (#9). 


The normalisations that correspond to these polynomials are as follows 
(for all 7 > 0): 


(U;, U;) = ||U;||? = | sin?(i + 1)0d0 = 47; (4.12) 
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I 


(Vin Vi) = VAI? = 2 f oos*(é + $)0.00 = (4.13) 


(Wi, Wi) = ||Will? 


I 


2 | sin?(i+ 4)0d0 = 7. (4.14) 
0 


(Remember that each of these three identities uses a different definition of the 
inner product (-, -), since the weights w(x) differ.) 


4.3. Orthogonal polynomials and best £2 approximations 


In Chapter 3, we characterised a best £.. (minimax) polynomial approxima- 
tion, by way of Chebyshev’s theorem, and this led us to an equioscillation 
property. Now we consider the best £2 polynomial approximation of a given 
degree, which leads us to an orthogonality property. 

The theorems in this section are valid not only for the inner product (4.2), 
but for any inner product (-, -) as defined by Definition 4.2. 


Theorem 4.1 The best L2 polynomial approximation p?(x) of degree n (or 
less) to a given (Lo-integrable) function f(x) is unique and is characterised 
by the (necessary and sufficient) property that 


(f — PR, Pn) =0 (4.15) 


for any other polynomial pp, of degree n. 


Proof: Write 
ie ees 


1. (Necessity) Suppose that, for some polynomial pp, 


(en >) #0. 


Then, for any real scalar multiplier ), 


2 | 2 


||f — @ + rp) en — pn 


= (en —2Xpn, en _ Non ) 
= (er ef) —2r (er Pn) +? (pn, Pn) 


<| 


B 
En 


2 
= 2d (en, Pn) +? [lpnll? 


BI? 
En 


for some small of the same sign as (en ; Dr): 


Hence p? + Apn is a better approximation than p? for this value of », con- 
tradicting the assertion that p? is a best approximation. 
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2. (Sufficiency) Suppose that (4.15) holds and that qn is any specified polynomial 
of degree n, not identical to p?. Then 


B 2 


| | 


If - gol? -|[F- pe 


En 
= (ef + (pt — am), €2 + (PB -an)) — (eR, ef ) 
— B B \ B B 

Pn Gn, Pn — Gn) +2( en, Pr — Qn 


> 0. 


2 
+0, from (4.15) 


B 
Pn — Qn 


Therefore || f — qn||” > \|f —p? ||’. 


Since gn is arbitrary, p? must be a best £2 approximation. It must also be 
unique, since otherwise we could have taken gn to be another best approxi- 
mation and obtained the last inequality as a contradiction. @@ 


Corollary 4.1A If {¢,} (¢; being of exact degree i) is an orthogonal poly- 
nomial system on [a,b], then: 


1. the zero function is the best Lo polynomial approximation of degree (n— 
1) to by on [a,b]; 


2. dn is the best Lo approximation to zero on [a,b] among polynomials of 
degree n with the same leading coefficient. 


Proof: 


1. Any polynomial p,_1 of degree n — 1 can be written in the form 


n-1 


Pn-1 = By Cidi. 


i=0 


Then 


n-1 
(gn —0, Pn—1) = (0 Sew) 


i=0 


n-1 


‘3 Ci (On ’ $i) 


i=0 


0 by the orthogonality of {d;}. 


The result follows from Theorem 4.1. 
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2. Let dn be any other polynomial of degree n having the same leading coefficient 
as gn. Then gn — on is a polynomial of degree n — 1. We can therefore write 


n—-1 


i=0 
and deduce from the orthogonality of {¢;} that 
(on, In — bn) = 0. (4.16) 
Now we have 
llgnll? — ball? = (ans an) — (Gn On) 


= (dn Ons In fn) 2 (on > dn — pn) 
= |lan — dnl”, using (4.16) 


Therefore ¢, is the best approximation to zero. @@ 


The interesting observation that follows from Corollary 4.1A is that every 
polynomial in an orthogonal system has a minimal £2 property — analogous 
to the minimax property of the Chebyshev polynomials. Indeed, the four kinds 
of Chebyshev polynomials T,,, Un, Vn, Wn, being orthogonal polynomials, 
each have a minimal property on [—1, 1] with respect to their respective weight 


functions 
1 5 1l+2 1-2 
’ T=", ) 
V1 — x? Vi-z l+a 


over all polynomials with the same leading coefficients. 


The main result above, namely Theorem 4.1, is essentially a generalisation 
of the statement that the shortest distance from a point to a plane is in the 
direction of a vector perpendicular to all vectors in that plane. 


Theorem 4.1 is important in that it leads to a very direct algorithm for 
determining the best Lz polynomial approximation p? to f: 


Corollary 4.1B The best L2 polynomial approximation pP of degree n to f 
may be expressed in terms of the orthogonal polynomial family {¢;} in the 


form 
p= Gey 
i=0 
where 


Cy = 7 eT 


(hi, bi) 
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Proof: For k=0,1,..., 


(fap, br) -_ (7- Soe. es) 


i=0 


= (f, on) — >> (bi, or) 


i=0 


= (Ff, be) — ce (be, Pe) 
= 0, by definition of cx. (4.17) 


Now, any polynomial p, can be written as 


Pr = oS didi, 
i=0 


and hence 


(f- pr, Pn) = Sra (tp, di) 


i=0 


= 0 by (4.17). 


Thus p? is the best approximation by Theorem 4.1. @@ 


EXAMPLE 4.1: ‘To illustrate Corollary 4.1B, suppose that we wish to determine 
the best Lo linear approximation pf to f(x) = 1— x? on [—1,1], with respect to 
the weight w(x) = (1 — a?)-2, In this case {T;(x)} is the appropriate orthogonal 
system and hence 

pi = coTo(a) + cr Ti(2) 
where, by (4.17), 


oi = —— 3 — 
ee ty ot (—22)- 41-2? )ede 
(fo, To) ie 


Substituting x = cos 6, 


co = = sin? oad = = | (1 — cos 20) dO = 3, 
T Jo 27m Jo 


a= = sin? 6 cos0d@ = es [4 sin? 0]? =0 
T Jo 7 
and therefore 
pi = $To(x) + 0Ti (x) = 5, 


so that the linear approximation reduces to a constant in this case. 
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4.3.1 Orthogonal polynomial expansions 


On the assumption that it is possible to expand a given function f(z) in a 
(suitably convergent) series based on a system {¢,} of polynomials orthogonal 
over the interval [a,b], 6, being of exact degree k, we may write 


Co 


f(c) =) agi(z), 2 € [a,0). (4.18) 


k=0 


It follows, by taking inner products with ox, that 
(f, bn) => ci (bis Oe) = Ck (bk, dr), 
k=0 


since (¢;, dx) = 0 for i 4 k. This is identical to the formula for cy, given in 
Corollary 4.1B. Thus (applying the same corollary) an orthogonal expansion 
has the property that its partial sum of degree n is the best £2 approximation 
of degree n to its infinite sum. Hence it is an ideal expansion to use in the 
£2 context. In particular, the four Chebyshev series expansions have this 
property on [—1,1] with respect to their respective weight functions (1 + 
x)+2(1—2)*2. 


We shall have much more to say on this topic in Chapter 5. 


4.3.2 Convergence in £2 of orthogonal expansions 


Convergence questions will be considered in detail in Chapter 5, where we 
shall restrict attention to Chebyshev polynomials and use Fourier series the- 
ory. However, we may easily make some deductions from general orthogonal 
polynomial properties. 


In particular, if f is continuous, then we know (Theorem 3.2) that ar- 
bitrarily accurate polynomial approximations exist in C[a, 6b], and it follows 
from Lemma 3.1 that these are also arbitrarily accurate in £2[a, b]. However, 
we have shown in Section 4.3.1 that the nth degree polynomial, P,,(x) say, 
obtained by truncating an orthogonal polynomial expansion is a best £2 ap- 
proximation. Hence (a fortiori) P,, must also achieve an arbitrarily small £2 
error ||f — P,,||, for sufficiently large n. This gives the following result. 


Theorem 4.2 If f is in Cla, b], then its expansion in orthogonal polynomials 
converges in Ly (with respect to the appropriate weight function). 


In Chapter 5, we obtain much more powerful convergence results for Cheb- 
yshev series, ensuring £2 convergence of the series itself for f in Lo[a, b] and 
Lo convergence of Cesaro sums of the series for f in C[a, b]. 
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4.4 Recurrence relations 


Using the inner product (4.2), namely 


b 
if, 9) = / w(2) f(2)9(x) de, 
we note that 


(f,9) = (9, f), (4.19) 
(cf, 9) = (f, £9). (4.20) 


The following formulae uniquely define an orthogonal polynomial system 
{¢;}, in which ¢; is a monic polynomial (i.e., a polynomial with a leading 
coefficient of unity) of exact degree 7%. 


Theorem 4.3 The unique system of monic polynomials {¢;}, with d; of exact 
degree i, which are orthogonal on [a,b] with respect to w(x) are defined by 


do(x) = 1, 
di(t) = z-a, (4.21) 
dn (x) = (x _ an) On—1(£) = bndn—2(£), 


where 


(Lbn-1 ; gn-1) (on=1 , n-1) 


7S SS, OE SS 
eS Gey a) (@n—21 On—2) 


Proof: This is readily shown by induction on n. It is easy to show that the 
polynomials ¢n generated by (4.21) are all monic. We assume that the polynomials 
go, 01, ---, @n—-1 are orthogonal, and we then need to test that ¢n, as given by 
(4.21), is orthogonal to ¢, (kK =0,1,...,n—1). 


The polynomial x¢, is a monic polynomial of degree k + 1, expressible in the 
form 


k 
won (2) = br+i(x) + >> cidi(2), 
i=l 
so that, using (4.20), 


(tbn-1 ’ ) =. (bn-1 ; xLpr) =0 (k <i 2), 
(thn 1; on 2) = (dn 1; Ln 2) = (don 1; On 1) 


For k < n — 2, then, we have 


(on, bk) = (tbn-1 , bk) — an (gn=1 , bk) — bn (on—2 ) bk) = 0, 
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while 


(on, On—2) = (€bn—-1, bn—2) — Gn (On-1, Gn—2) — bn (Gn—2, bn—2) 

= (bn-1, bn-1) — 0 — (bn-1, Gn-1) = 9, 

(dn, On—1) = (Lbn-1, Gn—-1) — An (On-1, Gn—1) — bn (Gn-2, Gn-1) 
= (ton-1, bn-1) — (tbn-1, bn—-1) —0= 0. 


Starting the induction is easy, and the result follows. @@ 


We have already established a recurrence relation for each of the four 
kinds of Chebyshev polynomials. We can verify that (4.21) leads to the same 
recurrences. 


Consider the case of the polynomials of the first kind. We convert T;,(x) 
to a monic polynomial by writing ¢9 = To, dn = 2'~"T, (n > 0). Then we 
can find the inner products: 


1 wT 
1 
Ty te = sir = f dé = 7, 
(To » To) [ Vv1— 2? 0 


1 wT 
x 
xTo, To) = a= f cos 6 dé = 0, 
( : 0) i; Vv 1 ae x? 0 
DME Ag a 
fared No a a= f cos? n6 dé = dn, 
( a Vv 1 = x? (0) 2 
1 Ty 2 7 
(2T,., In) = if a ae - ; cos @ cos” n6 d6 = 0. 
Therefore a; = 0, an = 0 (n > 1), and 
= (Gi, G1) _ (Ti, Ti) _ a 
(0; ¢0) bo) (To , To) To) o 
eer fe caeey cc eeee 
- (bn— 23 on— 2) (23-"T, 2, 29-5) 5 : 
So 
go = 1, 
On = w, 
b2 = @h1 — $90, 


on = Lon-1 = ton—2 (n > 2). 
Hence the recurrence (1.3) for Th. 


We may similarly derive the recurrences (1.6) for U,, and (1.12) for V,, and 
Wn, by using their respective weight functions to obtain the appropriate a, 
and by, (see Problem 5). 
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4.5 Rodrigues’ formulae and differential equations 


If {¢;} is a set of polynomials orthogonal on [—1, 1] with respect to w(x), with 
g; of degree 7, then 


i w(x) dbn(x)dn—1(x) dx = 0 (4.22) 


-1 


for any polynomial gn—1 of degree n — 1. 


Now suppose that r,() is an nth integral of w(x)dn(x), so that 
ry” (x) = w(2)n(2). (4.23) 


Then (4.22) gives, on integration by parts, 


0 =| 7) (2) qn—1(x) dx 


pei il we tava. ta) ak 


Hence, since g(x) = 0, it follows that 


[PY (ann (2) — 7°") (ahaha (2) #1 1)" rn (w) ah? (@)]_ = 0 


for any polynomial gn—1 of degree n — 1. 


Now ot) (2) = 0, since ¢, is of degree n; hence, because of (4.23), rn is 
a solution of the (2n + 1)st order homogeneous differential equation 


she (way rnto)) 2p, (4.25) 


An arbitrary polynomial of degree n—1 may be added to r,, without affecting 
the truth of (4.23) and (4.25). Hence we may without loss of generality arrange 
that rp(—1) = r’,(—1) =--- =r") (-1) = 0, when the fact that (4.24) is 
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valid for all gn—1 implies that r,(+1) = r/,(+1) =--- =r") (41) = 0, so 
that rp, satisfies the 2n homogeneous boundary conditions 


t(£1) = rl (£1) =--- = r-Y)(41) = 0. (4.26) 


One function satisfying (4.26), for any real a > —1, is 
(joys a yr: (4.27) 


If we then choose 
w(z) = (1 — 2?) (4.28) 


then (4.25) is satisfied, and ri” is of the form (4.23) with ¢n(x) a polynomial 
of degree n. 


Since ¢,, as defined by (4.22), is unique apart from a multiplicative con- 
stant, it follows from (4.25), (4.27) and (4.28) that (for a > —1) 


bn(x) = P) (x) := cp (ise eee (4.29) 


(1 — #?)% da” 


where c,, is a constant, defines a system of polynomials {PS (x)} orthogonal 
with respect to w(x) = (1 — 2?)® on [-1,1]. These polynomials are known 
as the ultraspherical (or Gegenbauer) polynomials, and the formula (4.29) for 
them is known as Rodrigues’ formula. 


It immediately follows that the Chebyshev polynomials of the first and 
second kinds are ultraspherical polynomials and, by comparing their leading 
coefficients with those in (4.29), we may readily deduce (see Problem 12) that, 


taking a = —4 and a= +3, 


ee ee ee 
T,(z) = (Qn)t — x") qt x”) ’ (4.30) 
TONG, = cpa dl iy n't 


(In the standard notation for Gegenbauer polynomials, as in Abramowitz 
and Stegun’s Handbook of Mathematical Functions (1964) for example, Po (x) 
is written as o2t8 (2), so that T,,(a) is proportional to () (a) and U,,(x) to 
co (x).) 

The well-known Legendre polynomials P,,(x), which are orthogonal with 
weight unity, are ultraspherical polynomials for a = 0 and are given by 


Pa), = a (1— 2)”, (4.32) 
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Note that the Chebyshev polynomials of third and fourth kinds are not 
ultraspherical polynomials, but only Jacobi polynomials. Their Rodrigues’ 
formulae are 


ve) = ET (152) {a2 F2) faa 


W,,(2) = ee =), ~~ {1 — x)" (=) }. (4.33b) 


From the general formula (4.29) it can be verified by substitution (see 


Problem 13) that PP (x) is a solution of the second-order differential equation 


(1 — 2)y" — 2(a4 1)ry’ + n(n + 2a+1)y = 0. (4.34) 

Thus T,,(x%), Un(x), Pn(a) are solutions of 
(1—2?)y"” —ay'+n?y=0 (a=-—3), (4.35a) 
(1—2")y"" — Say! +n(n+2)y=0 (a=), (4.35b) 


3 

(1 —a)y" —2ay'+n(n+1)y=0 (a=0), (4.35c) 
respectively. 

The differential equations satisfied by V,,(x) and W,,(a) are, respectively, 

(1 — 2?)y” — (Q2 —1)y’ +n(n+1)y = 0, (4.36a) 

(1 —2)y" — (224 ly’ +n(n+ ly = 0. (4.36b) 


4.6 Discrete orthogonality of Chebyshev polynomials 


It is always possible to convert a (continuous) orthogonality relationship, as 
defined in Definition 4.1, into a discrete orthogonality relationship simply by 
replacing the integral with a summation. In general, of course, the result is 
only approximately true. However, where trigonometric functions or Cheb- 
yshev polynomials are involved, there are many cases in which the discrete 
orthogonality can be shown to hold exactly. We give here a few of the dis- 
crete orthogonality relations that exist between the four kinds of Chebyshev 
polynomials. Further relations are given by Mason & Venturino (1996) (see 
also Problem 14). 


4.6.1 First-kind polynomials 


Consider the sum 
n+1 
s (9) = S- cos(k — $)@ = cos $6 + cos 20 +-+++ cos(n+$)0. (4.37) 
k=l 
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By summing the arithmetic series 
22 Hehehe ee), 


substituting z = e!’ and taking the real part of the result, it is easily verified 
(see Problem 4) that 

i 1)0 
aj Be (4.38) 
2sin 59 


and hence that s\) (0) vanishes when 6 = —, for integers r in the range 
n 


0<r<2(n+1). Further, we can see directly from (4.37) that 


s)(0)=n4+1, sY(2r) =-(n+1). (4.39) 
Now consider 
n+1 
Gp =) Ta) aes (OR 7 Gem) (4.40) 
k=1 
where x, are the zeros of T,41(z), namely 
(k— 4m 
= 6 a 4.41 
Lk COS UZ, O;, Aaet ( ) 
Then 
n+1 
aij = S- cos 10% cos JO 
k=1 
n+1 
= 4) [cos(i+ j)Ox + cos(i — j)6.] 
k=1 
Se ac (i+ j)r sng) (i—j)n 
2.{ [672 n+1 a n+1 
Hence 
aig = 0 (é x D3 1,9 < n), (4.42a) 
while, using (4.39), 
ai =s(n+1) (i #0; i<n) (4.42b) 
and 
ao0 = N+ 1. (4.42c) 


It follows from (4.42a) that the polynomials {T;(x), i = 0,1,...,n} are or- 
thogonal over the discrete point set {x,} consisting of the zeros of T,,41(2). 
Specifically, the orthogonality is defined for the discrete inner product 


n+1 


(u,v) = S- u(xp)v(rx) (4.43) 


k=1 
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in the form 
(%,7j)=0 G@AF; 45< nN), 
with (Tp, Zo) =n+1 and (T;, T;) = $(n+1) (0<i<n). 


This is not the only discrete orthogonality property of {T;}. Indeed, by 
considering instead of (4.37) the sum 


8) (0) = S~ cosk0 = $sinn6 cot $0 (n> 0) 
k=0 


(see Problem 4), where the double dash in ~” denotes that both first and 
last terms in the sum are to be halved, we deduce that 


for 0 < r < 2n, while 
s) (0) = s(?) (27) =n. 


If we now consider the extrema y, of T,(a), namely 
kr 
Yk = COS bk, oy = — (k = 0,1,...,n) (4.44) 
n 


(note that these {y,} are also the zeros of U,_-1(a) together with the end 
points +1), and define 


Mu 


biy = > Tilye)T; (ye), (4.45) 
k=0 
then we have 
by = 0 CAI 49S nN) (4.46a) 
by = $n (0<i<n) (4.46b) 
boo = ban = 1. (4.46c) 


In this case the inner product is 
1 


(u,v) = SY ulye)o(ye) (4.47) 


k=0 


n 


and we have again 
(T,Tj)=0 GAG 15S), 


but this time with (Zo, To) = (In, Tn) =n and (T;, T;) = $n, (0<i<n). 
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4.6.2  Second-kind polynomials 


In a similar way, we can establish a pair of discrete orthogonality relationships 
for the weighted second-kind polynomials {V1 — «? U;(x)} corresponding to 
the point sets {x,} and {y,} defined in (4.41) and (4.44)). 


Define 
a) = So (1 aR)Ui(wn)U;(en) (OS 4,5 <n) (4.48) 


where {x,} are zeros of T,41(x). Then we note that 


n+1 
ae = So sin(i + 1)0, sin(j +1)0, 
k=1 
n+1 
= $5 [cos(i — j)Ox — cos(i + j + 2)Ox)] 
k=1 
lL) (i —j)m sg) (@+j+2)r 
2 [en n+l a n+1 
Hence 
a =0 GAR 0<i,9 <n) (4.49a) 
and 
a =1(n4+1) (0<i<n), (4.49b) 
while 


a?) =n+1. (4.49c) 


Thus {V1 — 2? U;(x), i = 0,1,...,n} are orthogonal for the inner product 
(4.43). 


Similarly, considering the zeros {y,} of (1 — 2?)Un_1(z), 


Mu 


b= S~ (1 — yR)Ui(ye Us (ye) 


k=0 

m iN 
= sin(i + 1)dp sin(j + 1)dx. (4.50) 

k=0 

Then 

OO =0 GAG; ij<n-1) (4.51a) 
D0 =1n (0<i<n-1) (4.51b) 
ae =0 (4.51c) 


and {V1 — 2? U;(x), «= 0,1,...,2—1} are orthogonal for the inner product 
(4.47). 


84.6: DISCRETE ORTHOGONALITY OF CHEBYSHEV POLYNOMIALS 89 


4.6.3. Third- and fourth-kind polynomials 


Surprisingly, perhaps, the same discrete abscissae and inner products (4.43) 
and (4.47) provide orthogonality for the weighted third- and fourth-kind poly- 


nomials 
{V1l+c<Vi(x)}, {V1 —aW;(a)}. 


For we have 


n+1 
af) = S00 + 2e)Vi(ae)Vj(ex) (0 <i,5 <n) 
k=1 
n+1 
=2 oS cos(i + $)O% cos(j + $)O% 
k=1 
n+1 
— S“[cos(i + j+1)6, + cos(i— 7) Gx], 
k=1 
giving us 
a =0 GAS; 4,5 <n) (4.52a) 
a?) =n+1 (0<i<n), (4.52b) 
while 
i N 
of = So + yn)Vin)Vilyx) (OS 4,5 <n) 
k=0 
2 i 
= 2 > cos(t + 4) br cos(j + 4) br 
k=0 
ie N . : . . 
= S- [cos(¢ + 7 + 1)d,z + cos(t — 7) bx], 
k=0 
giving 
0 = 0 #5; i,j <n) (4.53a) 
b) =n (0<i<n). (4.53b) 


The same formulae (4.52)—(4.53) hold for a. and be, where 


n+1 n 


al) = S*(1 — ax) Wilwn)Wj(ae) and { = S~"(1— yx) Wilye) WV; (yn): 
= a (4.54) 
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4.7 Discrete Chebyshev transforms and the fast Fourier transform 


Using the values of a function f(x) at the extrema {y,} of T,(a), which are 
also the zeros of (1 — x?)U,_1(2), given as in (4.44) by 


k 
ye =cos— (k=0,...,n), (4.55) 
n 


we can define a discrete Chebyshev transform f(x), defined at these same 
points only, by the formula 


foe) := 2 S'Twdfos) (=0,....n). (4.56) 
j=0 


These values - (yx) are in fact proportional to the coefficients in the interpolant 
of f(yx) by a sum of Chebyshev polynomials — see Section 6.3.2. 


Using the discrete orthogonality relation (4.45), namely 


me GAS; ing<n), 
SS” Tilye)Ti(ye) = 4 gn (O<i=j<n), (4.57) 
k=0 n (i=j =Oorn), 


we can easily deduce that the inverse transform is given by 
7 MN A 
fy) = 2 So" Tew Fow) GF =0,.--.n). (4.58) 
k=0 


In fact, since 
jkr 
Tk (yj) = cos —— = Tj(yn), 
which is symmetric in j and k, it is clear that the discrete Chebyshev transform 
is self-inverse. 


It is possible to define other forms of discrete Chebyshev transform, based 
on any of the other discrete orthogonality relations detailed in Section 4.6. 


The discrete Chebyshev transform defined here is intimately connected 
with the discrete Fourier (cosine) transform. Defining 


(the zeros of sinn@) and 


g(0) = f(cos@), 9(8) = f(cosd), 
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the formula (4.56) converts to the form 
= /2 5, cos Ay ($;) (k=0,...,n). (4.59) 


Since cos@ and therefore g(@) are even and 27-periodic functions of 0, 
(4.59) has alternative equivalent expressions 


[kr _ i ai jkr ois = 


jan 
or hs 
. (kT i " ijkan (jn 
km\ _ fa jm EAST intey, 4.61 
i(=) te n (2 ( n, ,n) ( 61) 
or 


2n-1 


(F)=VEX exp Ut (=) (k=0,...,2n-1). (4.62) 


The last formulae (4.61) and (4.62) in fact define the general discrete 
Fourier transform, applicable to functions g(0) that are periodic but not nec- 
essarily even, whose inverse is the complex conjugate transform 


s(#) - VE D'o (() Ge am icny (463) 


k=-—n 


or 


2n-1 


(22 ae ES ue i(=) (j =0,...,2n—1). (4.64) 


4.7.1 The fast Fourier transform 


Evaluation of (4.56) or (4.58) for a particular value of k or j, respectively, 
requires a number O(n) of arithmetic operations; the algorithm described in 
Section 2.4.1 is probably the most efficient. If we require their values to be 
calculated for all values of & or j, however, use of this scheme would call 
for O(n?) operations in all, whereas it is possible to achieve the same results 
in O(nlogn) operations (at the slight cost of working in complex arithmetic 
rather than real arithmetic, even though the final result is known to be real) 
by converting the Chebyshev transform to the equivalent Fourier transform 
(4.62) or (4.64), and then computing its 2n values simultaneously by means 
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of the so-called fast Fourier transform (FFT) algorithm (Cooley & Tukey 
1965, Gentleman & Sande 1966). The required n+ 1 values of the Chebyshev 
transform may then be extracted. (The remaining n—1 computed results will 
be redundant, by reason of the symmetry of g and @.) 


While there are versions of this algorithm that apply when n is a product 
of any small prime factors (Kolba & Parks 1977, Burrus & Eschenbacher 1981, 
for instance), it is easiest to describe it for the original and most useful case 
where n is a power of 2; say n = 2. Then, separating the even and odd 
terms of the summation, (4.62) becomes 


2n—2 2n—-1 


i(= =) = re » pores Li zt o() + ay a ae ga uine idk (=) 
j fant j edd 
2ijk 29 
= EE 2H (2H) 
A n-1 oe : 
+ (exp us > exp ain o( SUT) ‘ (4.65a) 


while 


(Home *) = Vay ey (22) - 


7 -1 “28 c 
ika Qijkn ((2j+1)r 


j=0 
Now, if for 7 = 0,...,n —1 we define 


w (2) =o) ot n( 2) (32) 


we can further rewrite (4.65a) and (4.65b) as 


~[ka\ 1 . (2kr ee n( 
(=) = aa(= = )+3 a EXP — Ga (=). (4.66a) 


where the discrete Fourier transforms from g; to g, and from gz to gz each 
take a set of n values into another set of n values, whereas that from g to g 
takes 2n values into 2n. 
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Thus, once we have performed the two transforms of order n, it requires 
fewer than An further arithmetic operations (where kK denotes a small fixed 
integer) to compute the transform of order 2n. Similarly it requires fewer 
than 2 x kK = Kn operations to derive these two transforms of order n from 
four transforms of order n/2, fewer than 4 x kK 4 = Kn operations to derive 
these four transforms of order n/2 from eight transforms of order n/4, and so 
on. If n = 2”, therefore, (a transform of order 1 being just the identity and 
therefore trivial) the discrete Fourier transform of order 2n may be performed 
in (m+ 1) stages, each requiring fewer than Kn operations, so that the total 
number of operations is less than (m+ 1) Kn = O(nlogn), as claimed above. 


We do not propose to discuss in detail how this computation is best or- 
ganised, but refer the reader to the extensive published literature (Canuto 
et al. 1988, van Loan 1992, for instance). Reliable off-the-peg implementa- 
tions of a fast Fourier transform algorithm can be found in any comprehensive 
numerical subroutine library. 


4.8 Discrete data fitting by orthogonal polynomials: the Forsythe— 
Clenshaw method 


In this section we consider a least-squares/orthogonal polynomial method, 
in which Chebyshev polynomials fulfil what is essentially a supporting role. 
However, this is one of the most versatile polynomial approximation algo- 
rithms available, and the use of Chebyshev polynomials makes the resulting 
approximations much easier to use and compute. Moreover, the algorithm, in 
its Chebyshev polynomial form, is an essential tool in the solution of multi- 
variate data-fitting problems for data on families of lines or curves. 


We saw in Section 4.6 that an inner product may be defined on a discrete 
data set just as well as on a continuum, and in (4.43) we defined such an inner 
product based on Chebyshev polynomial zeros. However, we are frequently 
given a set of arbitrarily spaced data abscissae 


c=a, (k=1,...,m), (4.67) 
and asked to approximate in a least-squares sense the corresponding ordinates 
Y= Uk 


by a polynomial of degree n, where the number (n + 1) of free parameters is 
no more than the number m of given data — typically much smaller. 


Now 


(u,v) = S° weu(ep)v(ae) (4.68) 
k=1 


defines an inner product over the points (4.67), where {wx} is a specified set 
of positive weights to be applied to the data. From Corollary 4.1B, the best 
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polynomial approximation of degree n in the least-squares sense on the point 
set (4.67) is therefore 


p= S- cibi(2) (4.69) 
i=0 
where {¢;} are orthogonal polynomials defined by the recurrence (4.21) with 
the inner product (4.68) and where 


Gq = ———— 


(di, i) 


So weyndi (en) 
= =. (4.70) 


Wk [di(vx)]? 
k=1 


This is precisely the algorithm proposed by Forsythe (1957) for approximating 
discrete data y, at arbitrary points x, (k =1,...,n). 


The Forsythe algorithm, as we have described it so far, does not explicitly 
involve Chebyshev polynomials (or, for that matter, any other well-known set 
of orthogonal polynomials). However, if the data are distributed uniformly 
and very densely over an interval of x, [—1, 1] say, then we expect the resulting 
polynomials to be very similar to conventional orthogonal polynomials defined 
on the continuum [—1,1]. For example, if all the wz are equal to unity, 
then {¢,} should closely resemble the Legendre polynomials (orthogonal with 
respect to w(x) = 1), and if 


wr =(1- az)7? 


then {¢,} should resemble the Chebyshev polynomials of the first kind. In 
spite of this resemblance, we cannot simply use the Legendre or Chebyshev 
polynomials in place of the polynomials ¢; in (4.69) and (4.70), since on these 
points they are only approximately orthogonal, not exactly, and so we have 
to consider some other approach. 


The goal is a formula for p? based on Chebyshev polynomials of the first 
kind, say, in the form 


n 
/ 


pB(a)= >~ dh Tia), (4.71) 


i=0 


the new set of coefficients a” being chosen so that (4.71) is identical to (4.69). 
This form has the advantage over (4.69) that the basis {T;()} is independent 
of the abscissae (4.67) and therefore more convenient for repeated computa- 
tion of p?(x). (This is a very useful step in the development of multivariate 
polynomial approximations on lines or curves of data.) An efficient algorithm 
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for deriving the coefficients a” from the coefficients c; is due to Clenshaw 
(1959/1960); it makes use of the recurrence relations for Chebyshev polyno- 
mials as well as those for the discretely orthogonal polynomials {¢;}. We shall 
not give more details here. 


4.8.1 Bivariate discrete data fitting on or near a family of lines or 
curves 


Formula (4.71), which gives an approximation p? to data unequally spaced 
along one line, may readily be extended to much more general situations 
in two or more dimensions (Clenshaw & Hayes 1965). In two dimensions, 
for example, suppose that data are given at unequally-spaced and different 
locations on each of a family of lines parallel to the z-axis, say at the points 


(tre,ye), K=1,...,mig; €=1,...,me. 


We suppose too that all of these points lie within the square [—1, 1] x [—-1, 1]. 
Then the data on each line y = ye may be approximated, using Clenshaw’s 
algorithm, in the form (4.71), giving us a set of approximations! 


pe (a - > a” T(x), €=1,...,me. (4.72) 


The set of ith coefficients de), £=1,...,m2, may then be treated as data 
on a line parallel to the y-axis and may be approximated in a similar manner, 
for each i from 0 to n 1, giving approximations 


ne 


diy) = So de” T,(y). (4.73) 


j=0 


We thus arrive at the overall approximation 


5S ~ dn") T,(a)T;(y). (4.74) 


1=0 j=0 


If m1, M2, N41, Nz all = O(n), then the algorithm involves O(n*) operations— 
compared with O(n°) for a meshed data polynomial (tensor product) algo- 
rithm. It is important (Clenshaw & Hayes 1965) to ensure that there are data 
located close to x = +1 and y = +1, if necessary by changing variables to 
transform boundary curves into straight lines. 


1]f the number of data points on any line is less than the number of degrees of freedom, 
mye <1, then instead of approximating we can interpolate with a polynomial of minimal 


degree by requiring that a\e) =0 fori > my. 
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This algorithm has been extended further by Bennell & Mason (1991) to 
data on a family of curves. The procedure is to run secant lines across the 
family of curved lines and interpolate the data on each curve to give values 
at the intersections, which are then approximated by using the method just 
described. The algorithm involves only about twice as many operations as 
that for data on lines, which would appear very satisfactory. 


Other writers have developed similar algorithms. For example, Anderson 
et al. (1995) fit data lying ‘near’ a family of lines, using an iteration based on 
estimating values on the lines from the neighbouring data. 


Algorithms such as these have considerable potential in higher dimensions. 
An application to modelling the surfaces of human teeth has been successfully 
carried out by Jovanovski (1999). 


4.9 Orthogonality in the complex plane 


Formulae for Chebyshev polynomials in terms of a complex variable z have 
been given in Section 1.4; we repeat them here for convenience. 


Given any complex number z, we define the related complex number w to 

be such that 
z=}(w+w'). (4.75) 
Unless z lies on the real interval [—1, 1], this equation for w has two solutions 


w=ztiv2?-l, (4.76) 


one of which has |w| > 1 and one has |w| < 1; we choose the one with |w| > 1. 
Then we have: 


Tr(z) = $(w" +w~"); (4.77) 
wrth Rs wort 
U,(z) = ——————__; 4.78 
(2) = —_ (4.78) 
+4 4 1 
we'ztw 2 ywrttiy 
Va(z) = = ——____ 4.79 
(2) we + w-F w+ 1 ( ) 
1 1 
n+5 —n-5 n+1 —n 
wo'2—Ww 2 W —w 
Wr(z) = = (4.80) 
w2z—w 2 w-1 


For any r > 1, the elliptical contour EF, given by 


E, Se 2+ V2=1|=r} (r > 1) (4.81) 
has foci at z = +1, and is the image under (4.75) of the circle 


C, = {w:|w] =r} (4.82) 
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of radius r. 


The Chebyshev polynomials have useful orthogonality properties on the 
ellipse E,. In order to describe them, we need first to extend our definition 
(Definition 4.2) of the idea of an inner product to allow for functions that take 
complex values. Let 7 denote the complex conjugate of z. 


Definition 4.3 An inner product (-, -) is defined as a bilinear function of 


elements f,g,h,... of a vector space that satisfies the axioms: 
1. (f, f) is real and > 0, with equality if and only if f = 0; 
2. (f, 9) =(g, f) (note the complex conjugate); 
3. (f+g,h) = (f,h)+(g, h); 
4. (af, g) =alf, g) for any scalar a. 


(Hence (f , ag) =a (f, 9)-) 


This definition agrees with the earlier one if everything is real. 


Now define an inner product 


(f, 9) =f f(2)o@ lu) |ael (4.83) 


where p(z) is a weighting function (|(z)| is real and positive) and ¢ ---|dz| 
denotes integration with respect to arc length around the ellipse in an an- 
ticlockwise direction. This inner product corresponds to a norm ||-||, on E, 
defined by 


fle =f.) -$ FP le(2)| Ideal. (4.84) 
Then we can show, using this inner product, that 
0 (m # n) 
(Im, Tn) = § 20 (m=n=0) 
Lr(r2” + 7-2") (m =n > 0) 
F 1 
if (Zz) = Ving’ (4.85a) 


if u(z) = V1 - 2?, (4.85b) 
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if u(z) = : as =, (4.85¢) 
_ fo (mn) 
(Win, Wn) = jeans 4r-2"-1) (m=n> 0) 
if p(z) = : + =. (4.85d) 


Proof: Taking the first of these orthogonalities, for example, we have 


—— 4(w + w') = d (rel? ep te> 
dz = 3(1- w ?) dw = di(rel® —rte'®) da, 
Tm(z) = d(w™ w™) = Lapel 4 pa me): 


ies 2 = 2 a 2 

NE age ite 
ln(z)| | dz] = dé, 

Tm(z)Tn(2) as Ligne ne 4pm nei(mtn)a 


fe pomtne—i(m+n)o +4 po RS, 


reid — p-le-id 2 


Qn 
e'™? dd=0 (m0), 2n (m=O). 
0 
Hence we easily show that (Tm, Tn) = 0 form # n and is as stated form =n. @@ 


The other results (4.85) may be proved similarly (Problem 19), after noting 
that 


1 
jt- 27) = 7a |t-w’, 
wl 
1 2 
1 =] 
+= altel, 
1 2 
1—2| = ——|l-w)’. 
Hal = sia 


Note that || f||,, as defined in (4.84), as well as being a norm on the space 
of functions square integrable round the contour E;,, can be used as a norm 
on the space of functions which are continuous on this contour and analytic 
throughout its interior. This may be shown using the maximum-modulus 
theorem (Problem 21). 


Alternatively, we may define an inner product (and a corresponding norm) 
over the whole of the interior D, of the ellipse E,. 


Gigi / I H(2)9@ |ul2)| de dy, (4.86) 
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where z = x+iy. Remarkably, the Chebyshev polynomials are orthogonal with 
respect to this inner product too, defining p(z) for each kind of polynomial 
as in equations (4.85). 


Proof: ‘Take the first-kind polynomials, for example, when p(z) = 1/V1— 27. If 
a d(w +w') and w = se’, then z runs over the whole ellipse D, when s runs 
from 1 to r and @ runs from 0 to 27. We have 


w= 4(s+s *)cos6, 
y = 3(s—s ‘)sind, 
so that a(x.) 
zy 
dz dy = ds dé 
ee Bee 
with 
O(x,y) ds 3(1—s *)cos@ —4(s+s7')sin0 
O(s,0) o F014 57?) sing 3(s—s ')cos0 
2 1(s? + 2s cos 6 + 1)(s? — 2scos6 + 1) 
4 83 
while 
4s? 
OO ————————— 
(s? + 2s cos @ + 1)(s? — 2scos@ + 1) 
Thus, 


Go = f  F)aTE} al 2)| dr dy 
= ff ei 02 EB as a 
=f fou g(z)s_' ds dé 
af ap P* seyateao} 


But the inner integral is simply the inner product around the ellipse E;, which we 
have already shown to vanish if f(z) = Tm(z) and g(z) = Tn(z), with m F n. 
Therefore, the whole double integral vanishes, and (Im, Tn) =O formA#n. @@ 


Orthogonality of the other three kinds of polynomial may be proved in the 
same way (Problem 20). 


4.10 Problems for Chapter 4 


1. Verify that the inner product (4.2) satisfies the axioms of Definition 4.2. 
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2. Using only the properties of an inner product listed in Definition 4.2, 
show that the norm defined by (4.4) satisfies the axioms of Definition 3.1. 


3. If d, and w, are two polynomials of degree n, each of which is orthogonal 
to every polynomial of degree less than n (over the same interval and 
with respect to the same weight function), show that ¢,(x) and w(x) 
are proportional. 


4. Derive the summations 


n+1 


i 1)0 
S| cos(k — 4)0 = pullin 
ee 2sin 59 
n+1 

1— 1)6 
S anes = Owe 
es 2sin 39 


ee iW 1 1 
S- cosk@ = 5sinné cot 50 
MN 
S- sinkO = 4(1—cosné) cot $0. 


5. Using a similar analysis to that in Section 4.4, derive from (4.21) and 
the trigonometric formulae for U;(x), Vn (a) and W,,() the recurrence 
relations which are satisfied by these polynomials. Show that these 
relations coincide. 


6. Using the recurrence (4.21), obtain formulae for the monic (Legendre) 
polynomials of degrees 0, 1, 2, 3, 4, which are orthogonal on [—1, 1] with 
respect to w(a) = 1. 


7. If {¢,} is an orthogonal system on [—1, 1], with ¢, a polynomial of exact 
degree r, prove that the zeros of ¢,_, separate those of ¢,; that is to 
say, between any two consecutive zeros of ¢, there lies a zero of ¢,~1. 
(Hint: Consider the signs of ¢, as « — +00 and at the zeros of ¢,-1, 
using the recurrence (4.21).| 


8. A simple alternative to the recurrence (4.21) for the generation of a 
system of monic orthogonal polynomials is the Gram—Schmidt orthogo- 
nalisation procedure: 


Given monic orthogonal polynomials ¢9, ¢1, ..., @n—1, define 
gn in the form 


n—-1 
on(a) = a" + S- Cebp (x) 
k=0 
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and determine values of cz, such that ¢, is orthogonal to ¢o, 
$1, --+; On-1- 
Use this recurrence to generate monic polynomials of degrees 0, 1, 2 


1 1 
orthogonal on [—1, 1] with respect to (1+ a)” 2(1—2x)2. What is the 
key disadvantage (in efficiency) of this algorithm, compared with the 
recurrence (4.21)? 


9. By using the trigonometric formulae for T,,(#) and U,,(x), under the 
transformation x = cos 8@, verify that these Chebyshev polynomials sat- 
isfy the respective differential equations (4.35a), (4.35b). 


Show similarly that V,(a#) and W,,(x) satisfy the differential equations 
(4.36a), (4.36b). 


10. The second order differential equation (4.35a) 
(1—27)y" — ay’ +n*y =0 
has T;,(x) as one solution. Show that a second solution is V1 — #?U,_1(2). 
Find a second solution to (4.35b) 


(1 —2?)y” — 32y’ + n(n + 2)y = 0, 


one solution of which is U,(x). 


11. By substituting T,,(a%) = to+tiv7+---+t,x” into the differential equation 
that it satisfies, namely 


(1—2*)y" — ay’ + n°y =0, 
and equating coefficients of powers of x, show that t,_; = 0 and 


ty(n? —k7) +tpa(k+2)(k+1)=0, &=0,...,n—2. 


Deduce that 


tn-2m = (he ” 6 7 ™) ae? 


nm—-—m m 
where 
n\  nln—1)--*(n—k +1) 
k}) k! ; 
12. Writing 
d” 
aa — ¢?\nto — (1 — a”)"-"t*(A,a2” + lower degree terms), 
gr 


show that Ap41 = —(2n —r +2a)A, and deduce that 
An = (—1)"(2n + 2a)(2n + 2a — 1)--+ (n+ 2a). 


Hence, verify the formulae (4.30), (4.31) for T,(x), Un(x), determining 
the respective values of c, in (4.29) by equating coefficients of x”. 
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13. 


14. 


15. 


16. 


Verify that Pio) (x), given by (4.29), is a solution of the second order 
equation 


(1 —2?)y” —2(a+1)ay’ + n(n + 2a+1)y = 0. 
(Hint: Write 
Wn(a) =e, (1 — 2?)* PM) (2) = D*(1 — 27)" *, 
where D stands for d/da. Then derive two expressions for w,,;(«): 
Viyi(z) = DMA(L — a2)ho41 
= DY((1—2)(1— 22)", 
u (2) = D"*'D(1 Sgeyeront 
= -2(n+a@+ 1)D"" [21 — ¢?)"*4, 


differentiate the two products by applying Leibniz’s theorem, and equate 
the results. This should give a second-order differential equation for 
n(x), from which the result follows. ] 


Determine which of the following four systems of n + 1 weighted poly- 
nomials, 


{Ti(a)}, {V1—2? Ui(a)}, {V1 +aVil@)}, {vl —2xWi(x)} 


(0 < i < n) is discretely orthogonal with respect to which of the four 
following summations 


De. 38 = : 


zeros of Ty+41(x) zeros of (1 — x?)U,_1(z2) 


* / 


>) 


zeros of (1+ 2)V;,(x) zeros of (1 — «)W,,() 


(Pay particular attention to the cases i = 0 and i = n.) Find the values 
of (T;,7;), and similar inner products, in each case, noting that the 
result may not be the same for all values of 7. 


Using the discrete inner product (u, v) = 0, u(xp)u(ae), where {xz} 
are the zeros of T3(x), determine monic orthogonal polynomials of de- 
grees 0, 1, 2 using the recurrence (4.21), and verify that they are identical 
to {2!-*T,(x)}. 


If e 
fn(x) = xs ciTi1(2), (*) 
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where 
nm 


Ge = S~ (we) Tir (ae), 


n 
k=1 


and x, are the zeros of T,,(a), show that f(a.) = f(xx)- 
What does the formula (*) provide? 


What can we say about the convergence in norm of f,, to f as in — oo? 
17. Using the values of a function f(a) at the zeros {x} of T,(2), namely 


(2k + 1)r 


k=0,...,n- 
2n ( 0, »n 1), 


Lp = COS 


define another form of discrete Chebyshev transform by 


= V2 Yo Teles) f(s) (& =0,...,n—1). 
j=0 


Use discrete orthogonality to deduce that 


= VEL Teel f(tn) (GG =0,...,n-1). 


[See Canuto et al. (1988, p.503) for a fast computation procedure based 
on sets of alternate f values.] 


18. Using the values of a function f(a) at the positive zeros {x;,} of Ton(x), 
namely 

(2k+ 1) 
4n 


define another (odd) form of discrete Chebyshev transform by 


= i> Toryi(a;)f(aj) (k=0,...,n—1). 
j=0 


Lk = COS (k =0,...,n—1), 


Deduce that 


= VEL tale f (xr) (7 =0,...,n—1), 


and that this transform is self-inverse. [See Canuto et al. (1988, p.504) 
for a fast computation procedure. | 


19. Verify the orthogonality properties (4.85). 
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20. Show that {U,}, {V,} and {W,,} are orthogonal over the ellipse E, with 


21. 


respect to the inner product (4.86) and the appropriate weights. 


Evaluate (Tn, Tn), (Un, Un); (Vn, Vn) and (W,,, W,,) for this inner 
product. 


Prove that if A, denotes the linear space of functions that are analytic 
throughout the domain {z : |z+Vz?-1| < r} (r > 1), then ||-[l,, 
as defined by (4.84), has all of the properties of a norm required by 
Definition 3.1. 


CHAPTER 5 


Chebyshev Series 


5.1 Introduction — Chebyshev series and other expansions 


Many ways of expanding functions in infinite series have been studied. In- 
deed, the familiar Taylor series, Laurent series and Fourier series can all be 
regarded as expansions in functions orthogonal on appropriately chosen do- 
mains. Also, in the context of least-squares approximation, we introduced in 
Section 4.3.1 polynomial expansions whose partial sums coincide with best £L2 
approximations. 


In the present chapter we link a number of these topics together in the 
context of expansions in Chebyshev polynomials (mainly of the first kind). 
Indeed a Chebyshev series is an important example of an orthogonal polyno- 
mial expansion, and may be transformed into a Fourier series or a Laurent 
series, according to whether the independent variable is real or complex. Such 
links are invaluable, not only in unifying mathematics but also in providing us 
with a variety of sources from which to obtain properties of Chebyshev series. 


5.2 Some explicit Chebyshev series expansions 


Defining an inner product (f, g), as in Section 4.2, as 
1 
(f,9)= f w(a)sfle)ola) ae, (5.1) 
-1 


and restricting attention to the range [—1,1], the Chebyshev polynomials 
of first, second, third and fourth kinds are orthogonal with respect to the 
respective weight functions 


1 1l+2 1-2 
= ——— 1— 2? d : 5.2 
w(x) Rae Ve — ee (5.2) 


As we indicated in Section 4.3.1, the four kinds of Chebyshev series expansion 
of f(x) have the form 


f(z)~ SS, cii(2) (5.3) 
i=0 
where 
ci =(f, i)/(bi, Gi) (5.4) 
and 
o;(a) = T(x), Uj(x), Vi(x) or W; (ax) (5.5) 
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corresponding to the four choices of weight function (5.2). Values for (¢; , ¢;) 
were given in (4.11), (4.12), (4.13) and (4.14). 


In the specific case of polynomials of the first kind, the expansion is 


fla) ~ SZ eTi(w) = deoTo(a) + xTi(a) + e2To(x) +--+ (5.6) 
1=0 
where 7 
C= =/ — 2)? f(x)T;(a) da, (5.7) 


the dash, as usual, indicating that the first term in the series is halved. (Note 
the convenience in halving the first term, which enables us to use the same 
constant 2/7 in (5.7) for every 7 including 7 = 0.) 


There are several functions for which the coefficients c; in (5.6) may be 
determined explicitly, although this is not possible in general. 


EXAMPLE 5.1: Expansion of f(x) = V1 —<2?. 


Here 
a 1 T 
—q = / T;(x) dx =) cos 76 sin 6 dé 
2 a7 0 
= if [sin(i + 1)6 — sin(i — 1)6] dé 
) 
af cos(é =p. cos(é +1)0 G>1) 
2 i-1 i+1 3 
_ ( 1)? 1 7 ( 1)? 1 
2 i-—1 i+] 
and thus i 
Cok = — Tage ay? 2h-2 = 9 (Kh Sy 2353): 
Also 
co = 4/n. 
Hence, 
4 ety, Tor (x) 
Te ie) Derr 
k=0 
4 
= = ($To(x) = $T>(x) = T(x) = = 16(2) paces -) (5.8) 
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EXAMPLE 5.2: Expansion of f(x) = arccos x. 


This time, 
T 1 
3% = | (1 — 2?)72 arccos x T;(x) dx 
= 
= i 6 cos 16 dé 
0 
= oe -f" sin 26 do (i>1) 
i ' . i 
ae el 
= | + 
i i ‘ 
jet 
= e : 
so that ; 
=0 = k=1,2 
C2k » C2k-1 (2k — 1)? ( p28) 
Also 
co = 7 
Hence, 


TT 4 = Tor-1(2) 
arccos x ~ 3 lolz) —— Se ———_— 


= <To(x) = 


EXAMPLE 5.3: Expansion of f(x) = arcsin x. 
Here 
1 


CG = 7 (1 —2?)7? arcsin x T;(a) dx 


-1 


= | (= ~ 8) cos i0 a6 
10) 2 


= [ie cost (= - ) dé. 
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Now 


n/2 
a Cok = ‘| @ cosk(m — 26) d@ 
2 —1n/2 


w/ 
= tf ; @ cos 2k dd 


—1/2 
=0 


(since the integrand is odd), while 


Br m/2 
Boekel = Pes ¢ [cos(k — 4)m cos(2k — 1) + sin(k — 4) sin(2k — 1)¢] de 


Lye ie # sin(2k — 1)¢dd 
0) 


— 9f_1)*-2 ¢ cos(2k+1)¢ — sin(2k — 1)¢ m2 
= ae) - %—1 (2k — 1)? I, 
~ 2 
~ (2k—1) 
Hence, 
arcsin x ~ e 3 eee (5.10) 


Note that the expansions (5.9) and (5.10) are consistent with the relation- 
ship 


T : 
arccos © = a7 arcsin @. 


This is reassuring! It is also clear that all three expansions (5.8)—(5.10) are 
uniformly convergent on [—1, 1], since |7;(x)| < 1 and the expansions are 
bounded at worst by series which behave like the convergent series )>}° 1/k?. 
For example, the series (5.10) for arcsina is bounded above and below by its 
values at +1, namely 


CO 


~ oa ae 


Ce Al 


Since the series is uniformly convergent, the latter values must be +7/2. 


The convergence of these examples must not, however, lead the reader to 
expect every Chebyshev expansion to be uniformly convergent; conditions for 
convergence are discussed later in this chapter. 


To supplement the above examples, we list below a selection of other 
explicitly known Chebyshev expansions, with textbook references. Some of 
these examples will be set as exercises at the end of this chapter. 
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e From Rivlin (1974) 


4— p—1 L2n-1(2) 
k=1 
2 4— p_1 T2n(2) 


1 Fe 
a ee ss (a— +/a? — 1)?*T,(z) (a? > 1), (5.18) 


a av ar — k—=0 
: 2 a Ve aTT2) (@>1). (6.14 
~~ a-— Va? — "(x : ’ 
amie ar, Fay " 


e From Snyder (1966) 


T oo ; y2ktl : 
arctant ~ a Shay Leetile) (tin [0, 1)) (5.15) 
where « = Wee v= tan a (5.16) 
(/2—1)t+1 16 
sin zr ~ 21 )* Jon 41(z)Tors1 (2) (5.17) 


where J;(z) is the Bessel function of the first kind, 


ew 2 Do Fale) (5.18) 
sinh zz ~ 29 fassl) (z)Tops1(a), (5.19) 
cosh za ~ 2 5 Foa(2)Tae(a), (5.20) 


where I;,(z) is the modified Bessel function of the first kind, 


“VEE (3 — 2V2)'T#(«) (« in [0, 1), (5.21) 


(x in [0, 1)), (5.22) 
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2 / : 
where 6() is the ‘Dirac delta function’ with properties: 


d(2) = 0 for x £0, 


6(a) da = 1 for e > 0, 


/ (0) fle) de = F(0). 


(The expansion (5.23) obviously cannot converge in any conventional 
sense.) 


e From Fox & Parker (1968) 


arctan © 
~ > dap T oz (x (5.24) 
sad = 1) 2s+1 
h 
where agp = ae ee 71 


5.2.1 Generating functions 


At least two well-known Chebyshev series expansions of functions involve a 
second variable (as did (5.17)—(5.20)), but in such a simple form (e.g., as 
a power of u) that they can be used (by equating coefficients) to generate 
formulae for the Chebyshev polynomials themselves. For this reason, such 
functions and their series are called generating functions for the Chebyshev 
polynomials. 


e Our first generating function is given, by Snyder (1966) for example, in 


the form = 

F(u, 2) =e cos(uy/1 — 22) = S~ —Ta(z) (5.25) 
n! 
n=0 
which follows immediately from the identity 
u(cos 0+isin8)] __ = un 
Re[e J= S- 1 COS no. (5.26) 
n=0 


Although easily derived, (5.25) is not ideal for use as a generating func- 
tion. The left-hand side expands into the product of two infinite series: 


3 “ T,,(z) =e" cos(uy 1 — z?) = ~ ui ee ee 
n=0 i=0 al j=0 (23)! 
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By equating coefficients of wu”, multiplying by n! and simplifying, it 
is not difficult to derive the formula, previously quoted as (2.15) in 
Section 2.3.2, 


Te)= 0 or (5) (;) yn 2k (5.27) 


k=0 2j 


where |n/2| denotes the integer part of n/2. However, although it is 
a compact expression, (5.27) is expensive to compute because of the 
double summation. 


e A second and much more widely favoured generating function, given in 
Fox & Parker (1968), Rivlin (1974) and Snyder (1966), is 


Pian) = > = So Talxyu" (lul <1) (5.28) 


We follow the lead of Rivlin (1974) in favouring this. To obtain the 
coefficients in T;,(x), we first note that 


1 


F(u, 52) =(1- ==: 


12 ee) 


5 (5.29) 
and for any fixed x in [—1,1] the function u(z — wu) attains its greatest 
magnitude on |u| < 4 either at u = $2 (local maximum) or at one or 
other of u = +3. It follows that 


—7Su(e-u)<q (ul <q, [el <1) 


and hence that the second factor in (5.29) can be expanded in a conver- 
gent series to give 


mE =e v—u)" = Soa, sy (5.30) 


for |u| < 5. On equating coefficients of u” in (5.30), 


-—1 —2 —k 
oar (") yerea("S Jerta tue’ ee 


Se ah he (aye(" > Plan (5.31) 


Pp 


where p = |n/2]|. It is now straightforward to equate coefficients of u” 
n (5.28), replacing x by x/2 and using (5.29)—(5.31), to obtain 


rata) = 5 -¥|("Z*) (°F ae ay 


k=0 
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where we interpret (cr) to be zero in case n — k — 1 < k (which 
arises when n is even and k = p= n/2). Since the polynomial equality 
(5.32) holds identically for |z| < 1, it must hold for all x, so that we can 


in particular replace x by 2x to give 


T,,(«) = > (=1)Fan-2k-1 2 (" . ) = (" ~ ‘ 7 ‘)| g?2k (5,33) 


k=0 


Simplifying this, we obtain finally 


Ln/2| 
T(t) = >~ aie moar (" A *) ak (1 >0). (5.34) 


Formula (5.34) is essentially the same as formulae (2.16) and (2.18) of 
Section 2.3.2. 


5.2.2 Approximate series expansions 


The above special examples of explicit Chebyshev series generally correspond 
to cases where the integrals (5.4) can be evaluated mathematically. However, 
it is always possible to attempt to evaluate (5.4) numerically. 


In the case of polynomials of the first kind, putting x = cos 6 in (5.7) gives 


9 us 1 20 
- =| f (cos 0) cosi6 dé = — f (cos 0) cos i6 dd, (5.35) 
T JO T Jo 
since the integrand is even and of period 27 in @. The latter integral may be 
evaluated numerically by the trapezium rule based on any set of 2n+1 points 
spaced at equal intervals of h = a/n, such as 
(k-})n 


0 = 0, =, k=1,2,...,2n41. 
n 


(With this choice, note that {cos 0;,} are then the zeros of T;,(x).) Thus 


1 27 1 Oon41 h " 
c= - | gi(0) dd = - | gi(0)d0~ — S~ gi(Or), (5.36) 
T Jo T Jo, Tee 
where g;(@) := f(cos@)cosi0 and where the double dash as usual indicates 


that the first and last terms of the summation are to be halved. But g;(@1) = 
gi(O2n41), Since g; is periodic, and g;(2a — @) = g;(@) so that g;(O2n41-z) = 
gi(0,). Hence (5.36) simplifies to 


2¢ 20 
foe (On) = — a i=0,... ; 
Ge gil 2 (cos 6,) cosi#,, (i =0,...,n), (5.37) 


n 
k=1 
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or, equivalently, 


Cy = S° f (ex) Ti(ae) (5.38) 


where {x,} = {cos 6;} are the zeros of T,(x). 


Formula (5.37) is what is commonly known as a ‘discrete Fourier trans- 
form’, and is a numerical approximation to the (continuous) Fourier transform 
(5.35). In fact, if the infinite expansion (5.6) is truncated after its first n terms 
(to give a polynomial of degree (n — 1)), then the approximate series coeffi- 
cients (5.37) yield the polynomial of degree (& — 1) which exactly interpolates 
f(a) in the zeros {xz} of T,(x). So this approximate series method, based 
on efficient numerical quadrature, is really not a series method but an inter- 
polation method. This assertion is proved and the ‘Chebyshev interpolation 
polynomial’ is discussed in depth in Chapter 6. The trapezium rule is a very 
accurate quadrature method for truly periodic trigonometric functions of 6, 
such as g;(@). Indeed, it is analogous to Gauss—-Chebyshev quadrature for 
the original (a-variable) integral (5.7), which is known to be a very accurate 
numerical method (see Chapter 8). (On the other hand, the trapezium rule 
is a relatively crude method for the integration of non-trigonometric, non- 
periodic functions.) Hence, we can justifiably expect the Chebyshev interpo- 
lation polynomial to be a very close approximation to the partial sum (to the 
same degree) of the expansion (5.6). Indeed in practice these two approxima- 
tions are virtually identical and to all intents and purposes interchangeable, 
as long as f is sufficiently smooth. 


In Chapter 6, we shall state results that explicitly link the errors of a 
truncated Chebyshev series expansion and those of a Chebyshev interpolation 
polynomial. We shall also compare each of these in turn with the minimax 
polynomial approximation of the same degree. The interpolation polynomial 
will be discussed in this way in Chapter 6, but we give early attention to the 
truncated series expansion in Section 5.5 below. 


5.3 Fourier—Chebyshev series and Fourier theory 


Before we go any further, it is vital to link Chebyshev series to Fourier series, 
since this enables us to exploit a rich field as well as to simplify much of the 
discussion by putting it into the context of trigonometric functions. We first 
treat series of Chebyshev polynomials of the first kind, for which the theory 
is most powerful. 


Suppose that f(x) is square integrable (£2) on [—1, 1] with respect to the 
weight function (1 — 2)~?, so that 


/ (1 — 2?)~2 f(x)? da (5.39) 


1 
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is well defined (and finite). Now, with the usual change of variable, the 
function f(x) defines a new function g(@), where 


g(9) = f(cos0) (0<0A<7). (5.40) 


We may easily extend this definition to all real 6 by requiring that g(@+ 27) = 
g(@) and g(—@) = g(@), when g becomes an even periodic function of period 
27. The integral (5.39) transforms into 


if * 4(0)? 00, 


so that g is £2-integrable with unit weight. Thus, g is ideally suited to ex- 
pansion in a Fourier series. 


The Fourier series of a general 27-periodic function g may be written as 


Co 


g(9) ~ 4a9 + S_ (ax cos kO + bj, sin k8) (5.41) 
k=1 


ay ha oe 
aif (0) cos k0 a6, == f g(6)sink0d, (k =0,1,2,...). 


—t —w 

(5.42) 
In the present case, since g is even in 6, all the by coefficients vanish, and the 
series simplifies to the Fourier cosine series 


g(8) ~ S> ax cos k8 (5.43) 
k=0 
where st 
ap = =| g(0) cos k6 dé. (5.44) 
T JO 
If we now transform back to the x variable, we immediately deduce that 
= L 
f(x) ~ So axTe (2) (5.45) 
k=0 
where ‘ 
2 
an = =| (1 — x?)~? f (x) Tp (2) de. (5.46) 
WT J-1 


Thus, apart from the change of variables, the Chebyshev series expansion 
(5.45) is identical to the Fourier cosine series (5.43) and, indeed, the coef- 
ficients a, occurring in the two expansions, derived from (5.44) and (5.46), 
have identical values. 
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5.3.1 L£2-convergence 


A fundamental property of the Fourier series of any £2-integrable function 
g(@) is that it converges in the £2 norm. Writing the partial sum of order n 
of the Fourier expansion (5.41) as 


n 


(S19)(0) = $a0 + Y “(ax cos kO + by sin kO), (5.47) 
k=1 
this means that 
Ilg - SF gl|; - / [9(0) — (S*'g)(0)]? dd + 0 as n > oo. (5.48) 


Lemma 5.1 The partial sum (5.47) simplifies to 


1 [7 sin(n + 4)t 1 [7 
F 2 
je = t+ 9) ——— 2 at = — t + 0)W,,(cos t) dt, 
(SEN) = 5. f 9+) TE atm 3 | aft + OWaleast 
(5.49) 
where W,,(x) is the Chebyshev polynomial of the fourth kind. 
This is the classical Dirichlet formula for the partial Fourier sum. 
Proof: It is easily shown that 
ee i +)t 
S~ cos kt = cates 21 (5.50) 


1 
F=0 sin st 


Substituting the expressions (5.42) for a, and by in (5.47), we get 


(S'g)(0) = es | g(t) dé + = ae g(t)(cos kt cos kO + sin kt sin k@) dt 
k=1/—™ 


1 [7 it 
oS arf 9a To f aiecosett— aya 


1 is ! 
= =O D cos k(t — 0) dt 


a “ 
=- t+0 kt dt 
= | _alt+0) So" cos 


k=0 


a sin(n + 4)t 
/ g(t + gy dt 
alee sin st 


= ie g(t + 0)W,, (cos t) dt 


as required. @@ 
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In the particular case of the function (5.40), which is even, the partial sum 
(5.47) simplifies to the partial sum of the Fourier cosine expansion 


(SF 9)(0) = (SF@g) -> ax, cos k0. (5.51) 


This is identical, as we have said, to the partial sum of the Chebyshev series, 
which we write as 


= SX agTi(2). (5.52) 
k=0 


From (5.48) we immediately deduce, by changing variables, that 


/ (1 — x?)~2[f(x) — (ST f)(x)|? dz + 0 as N — 00, (5.53) 


nT 


provided that f(x) is £2 integrable on [—1,1] with weight (1 — 22)~?. Thus 
the Chebyshev series expansion is £2-convergent with respect to its weight 


function (1 — 2?)~?. 


We know that the Chebyshev polynomials are mutually orthogonal on 
[—1, 1] with respect to the weight (1 — x2)~2; this was an immediate conse- 
quence (see Section 4.2.2) of the orthogonality on [0,7] of the cosine functions 


if cosiécosj9d9=0 (i 47). 
0 


Using the inner product 


(fey fe) = / (1 — 22)" f(a) fole) de, (5.54) 


a4 
so that 
ae Bigs t (5.55) 


we find that 
(fF-SEF, F- Saf) =F, FY - 24S FPP + (SEE SAS) 


/ 


= ||f? -2 $0 an (Te, f) + $05 (To, To) + 
k=0 


+S ag (Te, Tr) 


k 
(from (5.52)) 


Rig - Me - 
= (f=2 S- eZ aK + se aes 
k=0 k=0 
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(from (5.55) and (4.11)) 
= IFIP - 5 DO a. 
k=0 


From (5.53), this expression must tend to zero as n — oo. Therefore ue az 


is convergent, and we obtain Parseval’s formula: 
ep 2 2 2 2 5 9\—1 2 
Yo ae == N FIP == [| - 27)? f(a)? de. (5.56) 
= T mw Jy 


The following theorem summarises the main points above. 


Theorem 5.2 If f(x) is Lo-integrable with respect to the inner product (5.54), 
then its Chebyshev series expansion (5.45) converges in Ly, according to 
(5.53). Moreover the infinite series Se 24 az is convergent to 2n—!||f\|? 
(Parseval’s formula). 


It is worthwhile at this juncture to insert a theorem on Fourier series, 
which, although weaker than the £2-convergence result, is surprisingly useful 
in its own right. We precede it with a famous inequality. 


Lemma 5.3 (H6lder’s inequality) Ifp > 1, q>1 and 1/p+1/q=1, and 
if f is Lp-integrable and g is Ly-integrable over the same interval with the 
same weight, then 


(Ff; 9) S Wfllp llglla- 
Proof: See, for instance, Hardy et al. (1952). @@ 


From this lemma we may deduce the following. 


Lemma 5.4 Jf 1 < pi < po and f is Ly,-integrable over an interval, with 
respect to a (positive) weight w(x) such that [ w(x) da is finite, then f is 
Ly, -integrable with respect to the same weight, and 


IIfllp, < CMF lly. 


where C' is a constant. 


Proof: In Lemma 5.3, replace f by |f|?!, g by 1 and p by p2/pi, so that q is 
replaced by p2/(p2 — pi). This gives 


ES SW esas: pyeasen) 
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or, written out in full, 


fe@ireor aes (f were ae) (f we) a) 


and therefore, raising this to the power 1/pi, 


IIfllp, S Cllfllys 
where C = (fw(x)dx)’? ”', @@ 


We can now state the theorem. 


Theorem 5.5 If g(0) is Lo-integrable on [—1, 7], then its Fourier series ex- 
pansion converges in the Ly norm. That is: 


‘a |9(@) - (S2"9)(8)| dd—- 0asn—- oo. 


Th 


Proof: By Lemma 5.4, 
|o- Seal], <elfo—srol, 


with C' a constant. Since a Fourier series converges in £2, the right-hand side tends 
to zero; hence, so does the left-hand side, and the result is proved. @@ 


5.3.2 Pointwise and uniform convergence 


So far, although we have established mean convergence for the Chebyshev 
series (4.24) in the sense of (5.53), this does not guarantee convergence at 
any particular point x, let alone ensuring uniform (i.e., £..) convergence. 
However, there are a number of established Fourier series results that we can 
use to ensure such convergence, either by making more severe assumptions 
about the function f(x) or by modifying the way that we sum the Fourier 
series. 


At the lowest level, it is well known that if g(@) is continuous apart from 
a finite number of step discontinuities, then its Fourier series converges to g 
wherever g is continuous, and to the average of the left and right limiting 
values at each discontinuity. Translating this to f(x), we see that if f(a) is 
continuous in the interval [—1,1] apart from a finite number of step discon- 
tinuities in the interior, then its Chebyshev series expansion converges to f 
wherever f is continuous, and to the average of the left and right limiting 
values at each discontinuity’. Assuming continuity everywhere, we obtain the 
following result. 

lif g or f has finite step discontinuities, then a further problem is presented by the 
so-called Gibbs phenomenon: as the number of terms in the partial sums of the Fourier or 
Chebyshev series increases, one can find points approaching each discontinuity from either 


side where the error approaches a fixed non-zero value of around 9% of the height of the 
step, appearing to magnify the discontinuity. 
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Theorem 5.6 If f(x) is in C[—1,1], then its Chebyshev series expansion is 
pointwise convergent. 


To obtain uniform convergence of the Fourier series, a little more than 
continuity (and periodicity) is required of g(0). A sufficient condition is that 
g should have bounded variation; in other words, that the absolute sum of 
all local variations (or oscillations) should not be unbounded. An alternative 
sufficient condition, which is neater but perhaps more complicated, is the 
Dini-Lipschitz condition: 


w(d)logd + 0 as 6 > 0, (5.57) 
where w(6) is a modulus of continuity for g(@), such that 

\9(9 + 6) — g(4)| < w(d) (5.58) 
for all 0. The function w(d) defines a level of continuity for g; for example, 
w(d) = O(6) holds when g is differentiable, w(d) — 0 implies only that g 
is continuous, while the Dini—Lipschitz condition lies somewhere in between. 
In fact, (5.57) assumes only ‘infinitesimally more than continuity’, compared 


with any assumption of differentiability. Translating the Fourier results to the 
x variable, we obtain the following. 


Theorem 5.7 If f(x) is continuous and either of bounded variation or satis- 
fying a Dini-Lipschitz condition on |—1,1], then its Chebyshev series expan- 
sion is uniformly convergent. 


Proof: We need only show that bounded variation or the Dini-Lipschitz condition 
for f(x) implies the same condition for g(9) = f(cos@). The bounded variation is 
almost obvious; Dini—Lipschitz follows from 


|9(9 + 6) — g(9)| = |f(cos(@ + 5)) — f(cos 4)| 
< w(cos(6 + 6) — cos 6) 
< w(d), 


since it is easily shown that |cos(6 + 6) — cos@| < |6| and that w(6) is an increasing 
function of |6|. @@ 


If a function is no more than barely continuous, then (Fejér 1904) we can 
derive uniformly convergent approximations from its Fourier expansion by 
averaging out the partial sums, and thus forming ‘Cesaro sums’ of the Fourier 
series 


(of 9)(8) = —(SF9 + Sfg +--+ SF_19)(0). (5.59) 


Then (c7’g)(@) converges uniformly to g(@) for every continuous function g. 
Translating this result into the Chebyshev context, we obtain not only uni- 
formly convergent Chebyshev sums but also a famous corollary. 
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Theorem 5.8 If f(x) is continuous on [—1,1], then the Cesaro sums of its 
Chebyshev series expansion are uniformly convergent. 


Corollary 5.8A (Weierstrass’s first theorem) A continuous function may 
be arbitrarily well approximated on a finite interval in the minimax (uniform) 
sense by some polynomial of sufficiently high degree. 


Proof: This follows immediately from Theorem 5.8, since (a7 f)(a) is a polynomial 
of degree n which converges uniformly to f(x) asn— co, @@ 


5.3.3 Bivariate and multivariate Chebyshev series expansions 


Fourier and Chebyshev series are readily extended to two or more variables 
by tensor product techniques. Hobson (1926, pages 702-710) gives an early 
and unusually detailed discussion of the two-dimensional Fourier case and its 
convergence properties, and Mason (1967) was able to deduce (by the usual 
x = cos @ transformation) a convergence result for double Chebyshev series of 
the first kind. This result is based on a two-dimensional version of ‘bounded 
variation’ defined as follows. 


Definition 5.1 Let f(x,y) be defined on D:= {-1 <a<1;-l<y< 1}; 
let {x,} and {y,} denote monotone non-decreasing sequences of n+ 1 values 
with x9 = yo = —1 and ey = Yn = +1; let 


y= SS lf (@r, Yr) — f(tr—-1 — Yr—1)], 
r=1 


hig i= oS Fl Setearei) = f(@r-1 = Yn—r)| ; 
r=1 


Then f(x,y) is of bounded variation on D if %1 and Ne are bounded for all 
possible sequences {x,} and {y,} and for every n > 0. 


Theorem 5.9 If f(x,y) is continuous and of bounded variation in 


S:{-l<a<1;-l<y< lH}, 


and if one of its partial derivatives is bounded in S, then f has a double 
Chebyshev expansion, uniformly convergent on S, of the form 
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However, Theorem 5.9, based on bounded variation, is not a natural ex- 
tension of Theorem 5.7, and it happens that the use of the Dini—Lipschitz 
condition is much easier to generalise. 


There are detailed discussions by Mason (1980, 1982) of multivariate Cheb- 
yshev series, interpolation, expansion and near-best approximation formulae, 
with Lebesgue constants and convergence properties. The results are generally 
exactly what one would expect; for example, multivariate Lebesgue constants 
are products of univariate Lebesgue constants. Convergence, however, is a 
little different, as the following result illustrates. 


Theorem 5.10 (Mason 1978) If f(a1,...,¢n) satisfies a Lipschitz condition 
of the form 


N N 
S > w5(5;) [[ 386; — 0 as 6; +0, 
j=l 


j=l 


where w,(d;) is the modulus of continuity of f in the variable x;, then the 
multivariate Fourier series of f, the multivariate Chebyshev series of f and 
the multivariate polynomial interpolating f at a tensor product of Chebyshev 
zeros all converge uniformly to f as n; — oo. (In the case of the Fourier 
series, f must also be periodic for convergence on the whole hypercube.) 


Proof: The proof employs two results: that the uniform error is bounded by 


1 
on (<=) 


(Handscomb 1966, Timan 1963, Section 5.3) and that the Lebesgue constant is of 
order [[log(n; +1). @@ 


5.4 Projections and near-best approximations 


In the previous section, we denoted a Chebyshev series partial sum by SZ’f, 
the symbol $7 being implicitly used to denote an operator applied to f. In 
fact, the operator in question belongs to an important family of operators, 
which we term projections, which has powerful properties. In particular, we 
are able to estimate how far any projection of f is from a best approximation 
to f in any given norm. 


Definition 5.2 A projection P, mapping elements of a vector space F onto 
elements of a subspace A of F, has the following properties: 


1. P is a bounded operator; 
i.e., there is a finite constant C such that ||Pf\| < C||f\| for all f in F; 
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2. P is a linear operator; 


te, PArfi +Aofe) = AP fitrA2P fo, where \1, r2 are scalars and fi, 
fo are in F; 


3. P is idempotent; 
i.e, P(Pf) =Pf for all f in F. 
Another way of expressing this, writing P? f for P(Pf), is to say that 


Poe (5.60) 


The last property is a key one, ensuring that elements of the subspace A 
are invariant under the operator P. This is readily deduced by noting that, 
for any g in A, there are elements f in F such that g = Pf, and hence 


Pg = P(Pf) = Pf =9. 


The mapping S$? of C[—1,1] onto the space I, of polynomials of degree 
n is clearly a projection. (We leave the verification of this as an exercise 
to the reader.) In particular, it is clear that S/ is idempotent, since the 
Chebyshev partial sum of degree n of a polynomial of degree n is clearly that 
same polynomial. 


On the other hand, not all approximation operators are projections. For 
example, the Cesaro sum operator defined in (5.47) is not idempotent, since 
in averaging the partial sums it alters the (trigonometric) polynomial. Also 
the minimax approximation operator B,, from C[—1, 1] onto a subspace A, is 
nonlinear, since the minimax approximation to 1 f; + A2f2 is not in general 
MBrfi + A2Bnfe. However, if we change to the Lz norm, then the best 
approximation operator does become a projection, since it is identical to the 
partial sum of an orthogonal expansion. 


Since we shall go into detail about the subject of near-best approximations, 
projections and minimal projections in a later chapter (Chapter 7), we restrict 
discussion here to general principles and to Chebyshev series (and related 
Fourier series) partial sum projections. In particular, we concentrate on Loo 
approximation by Chebyshev series of the first kind. 


How then do we link projections to best approximations? The key to this 
is the fact that any projection (in the same vector space setting) takes a best 
approximation into itself. Consider in particular the setting 


F =C[-1,1], A=TIlL, = {polynomials of degree < n} Cc F. 


Now suppose that P, is any projection from F onto A and that B,, is the 
best approximation operator in a given norm ||-||, and let J denote the identity 
operator. Then the best polynomial approximation of degree n to any f in F 
is B,f and, since this is invariant under P,, 
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The error in the approximation P,, f, which we wish to compare with the error 
in B,,f, is therefore given by 


f-Paf = (U-Pr)f = (E-Pa)f-(U-Pa)(Bnf) = (I-Pa)(f—Bnf), (5-62) 
using the fact that I, P, and hence (I — P,,) are linear. (Indeed, (I — P,) is 
another projection, since (I — P,)? = I—2P,, + P? = I— Ph, so that (I— P,) 
is also idempotent.) 

In order to go further, we need to define the norm of a linear operator, 


which we do in precisely the same way as the norm of a matrix. We also need 
to be able to split up the norm of an operator applied to a function. 


Definition 5.3 (Operator norm) If T is a linear operator from a normed 
linear space into itself, or into another normed linear space, then the operator 
norm ||T'|| of T is defined to be the upper bound (if it exists) 


|Tfll 


|T|| = sup [= - (5.63) 
rz llfll 
or, equivalently, 
Z| = sup ||Tf\l. (5.64) 
If ll=1 
Lemma 5.11 
ITF < LAS. (5.65) 


Proof: Clearly ||T|| > ||Z'f|| /||f|| for any particular f, since ||T'|| is the supremum 
over all f by the definition (5.63). @@ 


We may now deduce the required connection between P,,f and B,f. 
Theorem 5.12 For P,, and B, defined as above, operating from F onto A, 
lf —Pafll < - Pall lf -Brfll, (5.66a) 
If —Pafll < A+ Pall If - Bf: (5.66b) 


Proof: Inequality (5.66a) follows immediately from (5.62) and (5.65). 


Inequality (5.66b) then follows immediately from the deduction that for linear 
operators P and Q from F onto A 


IP+QI = sup IP +Q)sT 
= sup IPS +z 
< sup (IPSI+ IQs) 
< sup ||Pfl| + sup [l/l 
fil= IF ll=1 
= ||PIl+ QI 
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Hence 
|Z — Pnll < ||ZI] + || Pall = 1+ ||Pnl]. © 


Both formulae (5.66a) and (5.66b) in Theorem 5.12 give bounds on the 
error || f — P,f|| in terms of absolute magnification factors || — P,|| or (1 + 
||P, ||) on the best error || f — B, f||. Clearly minimisation of these factors is a 
way of providing the best bound possible in this context. In particular Cheney 
& Price (1970) give the following definitions. 


Definition 5.4 (Minimal projection) A minimal projection is a projec- 
tion P, from F onto A for which ||P,|| (and hence (1+ ||Pnll|)) is as small as 
possible. 


Definition 5.5 (Cominimal projection) A cominimal projection is a pro- 
jection P,, from F onto A for which ||I — P,|| is as small as possible. 


Sometimes we are able to establish that a given projection is minimal 
(or cominimal) — examples of minimal projections include (in appropriate 
settings) the partial sums of Fourier, Taylor and Laurent series. However, 
even if a projection is not minimal, the estimates (5.66a) and (5.66b) are very 
useful. In particular, from (5.66b), the value of ||P,,|| provides a bound on the 
relative closeness of the error in the approximation P,,f to the error of the 
best approximation. Mason (1970) quantified this idea in practical terms by 
introducing a specific definition of a ‘near-best approximation’, reproduced 
here from Definition 3.2 of Chapter 3. 


Definition 5.6 (Near-best and near-minimax approximations) An ap- 
proximation fx(x) in A is said to be near-best within a relative distance p 
if 

If — fll < +) If — fall, 


where p is a specified positive scalar and f(x) is a best approximation. In 
the case of the Lo. (minimax) norm, such an f* is said to be near-minimax 
within a relative distance p. 


Lemma 5.13 If P,, is a projection of F onto AC F, and f is an element of 
F then, as an approximation to f, Py f is near-best within a relative distance 


I|Pnll- 


Proof: This follows immediately from (5.66b). @®@ 


The machinery is now available for us to attempt to quantify the closeness 
of a Fourier—Chebyshev series partial sum to a minimax approximation. The 
aim is to bound or evaluate || P,,||, and this is typically achieved by first finding 
a formula for P,,f in terms of f. 
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5.5 Near-minimax approximation by a Chebyshev series 


Consider a function f(a) in F = C[-1,1] (ie., a function continuous for 
—1 <a <1) which has a Chebyshev partial sum of degree n of the form 


(St f)(«) = S~ ceTi(2), Ck = =| On ae (5.67) 
k=0 =1 —& 


If, as in Section 5.3, we define 
(9) = f(cos@) 


then we obtain the equivalent Fourier cosine series partial sum 


n 


9 Tv 
(SF°g)(0) = S~ cycoskO, ce = = | g(8) cos k6 dd. (5.68) 
k=0 


The operator SF° can be identified as the restriction of the Fourier projection 
S¥ to the space C2,,¢ of continuous functions that are both periodic of period 
27 and even. Indeed, there is a one-to-one relation between f in C[—1,1] and 
gin C$, , under the mapping x = cos 0, in which each term of the Chebyshev 
series of f is related to the corresponding term of the Fourier cosine series of 


g. 
Now, from Lemma 5.1, we know that S* may be expressed in the integral 
form (5.49) 


(5 9)(0) = + / F petog net aN (5.69) 


wn 
27 Jon sin 5¢ 


From this expression, bounding g by its largest absolute value, we get the 
inequality 
(S29) (9)| < An W9lloo (5.70) 


where 


An = ao. val ad dt = 
2m Jn sin 5t 
1 (7 |sin(n+ 4)t 4 We 
2 - | a [=> Wn ()| a]. Bis 
™ JO sin xt Tw J_1 V1 — 2? 
Taking the supremum over 6 of the left-hand side of (5.70), 
ISraloo S An Igloo» (5.72) 


whence from (5.63) it follows that 


|S2'||_. < An (5.73) 
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and, a fortiori, since 


FC 
sup WSECAlas — guy USE tll < guy USE Alas 
gece Illoo gece. “Mdlloc ee. Wdlloc 


Qr,e Qr,e 


that 
Se hoo S Si llc S An: (5.74) 


As a consequence of the one-to-one relationship between every f(x) in 
C[—1, 1] and a corresponding (9) in C3,,,, it also immediately follows that 


IS loo = PSEC Ilo < An (on the space C[—1, 1)). (5.75) 


From Theorem 5.12 we may therefore assert that (92 f)(x) is near-minimax 
within a relative distance ,. So, how small or large is A,? Or, in other words, 
have we obtained a result that is really useful? The answer is rather interest- 
ing. 

The constant A, is a famous one, the Lebesgue constant, and it is not 
difficult to show that 


4 
An > < logn. (5.76) 


So A, tends to infinity with n, which seems at first discouraging. However, 
logn grows extremely slowly, and indeed 4,, does not exceed 4 for n < 500. 
Thus, although it is true to say that Sf becomes relatively further away 
(without bound) from the best approximation B,,f as n increases, it is also 
true to say that for all practical purposes $7 f may be correctly described as 
a hear-minimax approximation. Some values of \,, are given in Table 5.1. 


Table 5.1: Values of the Lebesgue constant 


More precise estimates than (5.76) have been derived by a succession of 
authors; for instance, Cheney & Price (1970) give the asymptotic formula 


4 
An = wy login + 1.2703... + O(1/n). (5.77) 
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5.5.1 Equality of the norm to \,, 


We have not yet fully completed the above analysis, since it turns out in fact 
that we may replace ‘<’ by ‘=’ in (5.73), (5.74) and (5.75). This does not 
improve the practical observations above, but it does tell us that we cannot 
find a better bound than that given by (5.71). 


To establish equality, it suffices to show that one particular function g(0) 
exists in C3, ,, and one value of @ exists in [0,7], for which 


|(S2°9)(8)| > An IIdlloo — 5 (5.78) 
with 6 arbitrarily small — for then we must have 
[Sr 9]5 = Se loo 2 An Ilgllc (5.79) 
and hence, from (5.72), 
Sn alloc = ISe alloc = An lige (5.80) 
and finally 
Se loo = WSe loo = Sell. = An: (5.81) 
Proof: First, define 
: 1 
gp(@) := sgn (mo) ; (5.82) 
sin 50 
where 
+1, «>0 
sgn(x) := 0, «=0 
-1, «<0. 
Then 
Ilgoll.o = 1: (5.83) 


Moreover gp is continuous apart from a finite number of step discontinuities, and 
is an even periodic function of period 27. It is now a technical matter, which we 
leave as an exercise to the reader (Problem 6), to show that it is possible to find a 
continuous function gc(@), which also is even and periodic, such that 


Igo —golh = f lgc(t) - gn(0)| at <e 
and such that ||go||,, is within € of unity, where ¢ is a specified small quantity. 


Then, noting that n is fixed and taking 6 as 0 in (5.69) 


1 af sin(n + 4)t 
S*gc)(0) = — fy a Ae 
(8i'90)(0) = =f get) 
1 /* sin(n + 5)t 1 [7 sin(n + 5)t 
=> t) — gp(t dt + — ¢t) ———_ dt 
rz | (9c) — 900) EE art 5 footy 
1 sin(n + 4)t 
Sao (gc (t) Syn @ tan dt+ An, from (5.71), 
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while 

Te fe sin(n + 4)t 1 |/sin(n + 4)t 

— t) — go(t)) ———— dt} <_ — |] —4 — 

x | (ac) gp(t)) net = oe gine __ lie gpl, 
= = [|Wall.o | | 
i on Nl oo Jc gD 1 
Shanty 

—(4n 


since |W,,(x)| has a greatest value of 2n + 1 (attained at x = 1). 
Thus : 
|(S°9c)(0)| = An — =(2n + Le 
and 
An [loco S An(L + €). 


For any small 6, we can then make e€ so small that (5.78) is satisfied at 06 = 0 by 
g=gc. @@ 


5.6 Comparison of Chebyshev and other orthogonal polynomial 
expansions 


The partial sum (5.47) of a Fourier series represents a projection from the 
space C$ onto the corresponding subspace of sums of sine and cosine func- 
tions, that is both minimal and cominimal (in the minimax norm). This may 
be shown (Cheney 1966, Chapter 6) by considering any other projection op- 
erator P from C$. onto the space of linear combinations of sines and cosines 
up to cosné and sin né, letting T) be the shift operator defined by 


(Ty f)(0) = f(9+A) for all 6 


and showing that 


af (TaPRANO)Ad = (SENO), (5.84) 


27 Ja 
Since ||7)||,, = |/Z_all,, = 1, we can then deduce that 
IIPlloo = WSi Moo» 
so that S is minimal. It follows likewise, since (5.84) implies 


1 Tv 


Qn Jn 


(T_)(I — P)T) f)(9) dd = (I — SP) F)(O), (5.85) 


that S* is cominimal. 
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Thus we can say that the partial sums of the Fourier expansion of a con- 
tinuous periodic function ‘converge faster’, in terms of their minimax error 
bounds, than any other approximations obtained by projection onto subspaces 
of trigonometric polynomials. 


Remembering what we have successfully done on many previous occasions, 
we might have supposed that, by means of the substitution x = cos 6, we could 
have derived from the above a proof of an analogous conjecture that the par- 
tial sums of a first-kind Chebyshev expansion of a continuous function on 
[—1,1] converge faster than any other polynomial approximations obtained 
by projection; that is, than the partial sums of an expansion in polynomials 
orthogonal with respect to any other weight. Unfortunately, this is not pos- 
sible — to do so we should first have needed to show that $* was minimal 
and cominimal on the space C$, , of even periodic functions, but the above 
argument then breaks down since the shift operator 7, does not in general 
transform an even function into an even function. 


The conjecture closely reflects practical experience, nevertheless, so that 
a number of attempts have been made to justify it. 


In order to show first-kind Chebyshev expansions to be superior to expan- 
sions in other ultraspherical polynomials, Lanczos (1952) argued as follows: 
Proof: The expansion of a function f(x) in ultraspherical polynomials is 


Co 


(ase PO @) (5.86) 
k=0 
with coefficients given by 
/ "(1 = 2)" f(a) PO (®) de 
a (5.87) 
/ (1 — 2)? [P (a) de 


iL 


Using the Rodrigues formula (4.29), this gives us 


fe = (5.88) 
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/ * f(e) (1— 02)" de 
-1 


—_——————— 
Ko f (1 —2?)*** de 


1 


(5.89) 


where Ko is the coefficient of the leading power x* in P{ (2). 
As k — oo, then claims Lanczos, the factor (1 — «?)**°® 
approaches a multiple of the delta function 6(x), so that 


in each integrand 


() 
(a) | £(0) 

An , 5.90 
* RK na 


Since we have not yet specified a normalisation for the ultraspherical polynomials, 
we may take them all to be monic polynomials ( Aes = 1), so that in particular 


= 
Pi 2) (a) = 2'-*T,(2). Then the minimax norm of the kth term of the expansion 
(5.86) is given by 


Je (5.91) 


Js 


— | £O) jp 
oo k! ke 


co 


But (Corollary 3.4B) P-® (2) = 2'-*T;(ax) is the monic polynomial of degree k 
with smallest minimax norm on [—1, 1]. Hence the terms of the first-kind Chebyshev 
expansion are in the limit smaller in minimax norm, term by term, than those of 
any other ultraspherical expansion. @®@ 


This argument is not watertight. First, it assumes that f™ (0) exists for all 
k. More seriously, it assumes that these derivatives do not increase too rapidly 
with & — otherwise the asymptotic form (5.90) cannot be justified. By use 
of formulae expressing the ultraspherical polynomials as linear combinations 
of Chebyshev polynomials, and by defining a somewhat contrived measure 
of the rate of convergence, Handscomb (1973) was able to find a sense in 
which the first-kind Chebyshev expansion converges better than ultraspherical 
expansions with a > -4, but was unable to extend this at all satisfactorily 
to the case where —1 <a < —4. Subsequently, Light (1978) computed the 
norms of a number of ultraspherical projection operators, finding that they 
all increased monotonically with a, so that the Chebyshev projection cannot 
be minimal. However, this did not answer the more important question of 
whether the Chebyshev projection is cominimal. 


Later again, Light (1979) proved, among other results, that the first-kind 
Chebyshev expansion of a function f converges better than ultraspherical 
1 


expansions with a > —5, in the conventional sense that 


f S- ch) Po 
k=0 


provided that f has a Chebyshev expansion )>, bgT with 


Il f - oe glee < for sufficiently large n, (5.92) 


[oe 


2* |b. | = A as k > oo. (5.93) 
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Equation (5.93) is, in effect, a condition on the smoothness of the function f 
sufficient to ensure that we cannot improve on the accuracy of the first-kind 
Chebyshev expansion by expanding in ultraspherical polynomials po for 
any a > —4 (and so, in particular, in Legendre polynomials or in second-kind 
Chebyshev polynomials). Light’s analysis, however, still does not exclude the 
possibility that we could get faster convergence to such a function f by taking 
O<a< -t, although we do not believe that anyone has yet constructed a 
function f for which this is the case. 


5.7 The error of a truncated Chebyshev expansion 


There are many applications of Chebyshev polynomials, especially to ordinary 
and partial differential equations, where we are approximating a function that 
is continuously differentiable, finitely or infinitely many times. If this is the 
case, then Chebyshev expansion converges very rapidly, as the following the- 
orems show. 


Theorem 5.14 If the function f(x) has m+ 1 continuous derivatives on 
[—1, 1], then | f(z) - 5? f{a)| =O(n—-™) for all x in [-1, 1]. 


We can prove this using Peano’s theorem (Davis 1961, p.70) as a lemma. 
Lemma 5.15 (Peano, 1913) Let L be a bounded linear functional on the 


space C™*1[a,b] of functions with m+ 1 continuous derivatives, such that 
Lpm = 0 for every polynomial pm in Im. Then, for all f € C™*1{a, 6], 


b 
Lf = / fO"*D (4) (t) dt (5.94) 
where : 
K(t) = L(.—#)f. (5.95) 


Here the notation (-)'’ means 


(c—t)P = { 0, oon (5.96) 


Proof: (of Theorem 5.14) 


Let f ec™t[-1, 1]. If $2’ f, as in (5.67), is the Chebyshev partial sum of degree 
n > m of f, then the operator Ln, defined for any fixed value x € [—1,1] by the 
relationship 


Inf = (Sn f)(x) — f(z), (5.97) 
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is a bounded linear functional on em =1, 1]. Since Spm = pm for every poly- 
nomial in I], it follows that Lnpm = 0 for every such polynomial. Using Peano’s 
theorem, we deduce that 


(ST f)(e) — f(a) = fh FO OKu(at) at (5.98) 
where 


Ka(a,t) = {ST (0-8) — (@- 8/7}. (5.99) 


We note that in (5.99) the operator S7 must be regarded as acting on (a—t)" as 
a function of z, treating t as constant; thus, explicitly, 97 (e—t)"? = Tyo ComTk (2) 


where : (x) 
2 a—t)”"T, (x 
Ckm = — —_—— dx 5.100 
mae f Se (5.100) 
or, writing x = cos@ and t = cos 4@, 
a peer 
Ky, (cos 6, cos ¢) = mi es Ckm Cos k@ — (cos 8 — cos a | (5.101) 
: k=0 
where 
>) % 
Ckm = = (cos 6 — cos ¢)”” cos k@ dé. (5.102) 
T So 


Now it can be shown that chm = O(k~™~') as k = oo. It follows that 


Sa(e— 2% -(@-2)T |=] D2 cemTe(2)| S$ SD leem| = O(n) 
k=n+1 k=n+1 


and hence finally, using (5.99) and (5.98), 


(St A(@) - F@)| = OM"). 
This completes the proof. @@ 


If f is infinitely differentiable, clearly convergence is faster than O(n—™) 
however big we take m. In some circumstances we can say even more than 
this, as the following theorem shows. 


Theorem 5.16 If the function f(x) can be extended to a function f(z) ana- 
lytic on the ellipse E,. of (1.44), where r > 1, then | f(a) _ ST f(a)| = O(r7") 
for all x in [-1,1]. 


Proof: Suppose that 
M =sup{|f(z)| : 2 € Ey}. (5.103) 


The Chebyshev expansion will converge, so that we can express the error as 


f()-(S8A@= Y= / (=v YER) W)Te(@) ay. (6.104) 


k=n+1 
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Using the conformal mapping of Section 1.4.1, with 


c=Z(€+E"), f(x) =9(8) =9(E") 


(so that |g(¢)| < M for r~* < |¢| <r), and remembering that integration around 
the unit circle C; in the €-plane corresponds to integration twice along the interval 
[—1,1] in the x-plane (in opposite directions, but taking different branches of the 
square root function), we get 


f(a) — (Sn f(z) = 


oo 


= = k —k k —k dn 
= a) ee , in a Me es 
ss -kgk , gk) AN re ae ee 
= ee im fom (ereeers) ; +f acon (7 +é-") 5 


— since all parts of the integrand are analytic between C, and C1/, 
ay = _ky dn 
=> af (nn * (Ee + €*)— 
In So ” 

k=n+1 r 


— replacing 7 by 77! in the second integral, and using g(7) = g(n7') 


1 Entity n-1 é n 7 n-1 dn 
~ Din a a Pr | eae ll 


where || = 1 when z € [—1, 1]. Therefore 


M 


CEST (5.106) 


|F@) - (ST. @)| < 


The Chebyshev series therefore converges pointwise at least as fast as r~". @@ 


5.8 Series of second-, third- and fourth-kind polynomials 


Clearly we may also form series from Chebyshev polynomials of the other 
three kinds, and we would then expect to obtain results analogous to those 
for polynomials of the first kind and, in an appropriate context, further near- 
best approximations. First, however, we must consider the formation of the 
series expansions themselves. 


5.8.1 Series of second-kind polynomials 


A series in {U;(x)} can be found directly by using orthogonality as given by 
(5.1)—(5.4). If we define a formal expansion of f(x) as 


f(a) ~ de Uila); (5.107) 


i=0 
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then i 
ff = f 1-2) (Ula) ax/ Ws, 0) 
=a 
where 
1 BH 
0) = i (1 — 22) #U;(2)? de 
—1 
= sin?(i + 1)0.d0 
0 
= 
Thus 
1 fi i 
ef = 2 f (1 2)¥ F(2)Ui(a) da (5.108a) 
WT SJ-1 
ae | annie Hae Fdbah) ae. (5.108b) 
T Jo 


For any given f(x), one of these integrals may be computed analytically or 
(failing that) numerically, for each 7, and hence the expansion (5.107) may be 
constructed. 


It is worth noting that from (5.108b) we can get the expression 


1 Tv 
eos ~ | {cos i@ — cos(t + 2)6} f (cos 0) dé 
T Jo 
= 4f{eF —cF,5}, (5.109) 


where {c/ } are the coefficients of the first-kind Chebyshev series (5.6) of f(z). 
This conclusion could equally well have been deduced from the relationship 
(1.7) 

Uy, (a) — Un—2(x) = 2T, (2). 


Thus a second-kind expansion can be derived directly from a first-kind ex- 
pansion (but not vice versa). 


Another way of obtaining a second-kind expansion may be by differenti- 
ating a first-kind expansion, using the relation (2.33) 


Tela 10 gy AG) 


For example, the expansion (5.18), for z = 1, 


e* ~ Ip(1) + 2S H(1)Ti(2) 
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immediately yields on differentiation 
e 2S (64 Ihiyi (1)Ui(2), (5.110) 
i=0 


where J; is the modified Bessel function. 


(Note that we have )> and not S~’ in (5.110) — that is, the Up coefficient 
is not halved in the summation. It is only in sums of first-kind polynomials 
that this halving is naturally required.) 

Operating in reverse, we may generate a first-kind expansion by integrat- 
ing a given second-kind expansion. In fact, this is a good approach to the 
indefinite integration of a given function, since it yields a first-kind expansion 
of the integral and hence its partial sums are good approximations in the L. 
sense. We shall discuss this in more depth later. 


It can also sometimes be advantageous to weight a second-kind expansion 
by /1—x?. For example, the expansion 


V1—2? f(a) ~ >> cl /1- 270; (2), (5.111) 
1=0 


where cY are defined by (5.108a) or (5.108b), can be expected to have good 
convergence properties provided that f(a) is suitably smooth, since each term 
in the expansion has a minimax property among polynomials weighted by 
V1 — 2?. 


5.8.2 Series of third-kind polynomials 


A function may also be directly expanded in third-kind polynomials in the 
form 


f(a) ~ So ef Vi(a). (5.112) 
1=0 


Now if x = cos6@ then 1) 
cos(i + 5)@ 
LoS 

”) cos $0 


and 
dx = —sin@d0 = 2sin $6 cos 50d0. 


Hence 
fi,(1 +2)? (1 — 2)72V, (x) f(a) de 
fi, +2)? (1 —2)-2Vi(2)? da 
Jo 2cos $8 cos(i + $)0 f (cos 0) dd 
Jo 2cos $0 cos2(i+4)0d0 - 


cY 


4 
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Thus 

= - [boos i0 + cos(i + 1)0} f (cos 0) dd = {ce} +c}, ,} (5.113) 
(which is consistent with (1.20)); the expansion coefficients may hence be 


calculated either directly or indirectly. 


For example, suppose 


so that f(cos@) = sin $6. Then 


1 Tv 
cY a sin @ cos(i + )0.d0 
0 


TT 


= ~ | 2sin 2d cos(2i + 1)bdd 
0 


[sine + 3)$ — sin(2i — 1)4] dé. 
0 


Thus 


iy ape 1 4 1 
Voss cal eee ciay Cet: 114 
“ 7 C= ss} 7 (2i— 1)(21 + 3)’ ene) 


and we obtain the expansion 
1 )? 

2-2(1-—a)2 ~ ns : 5.115 

*(1-2)} or emer (5.115) 


In fact, any third-kind expansion such as (5.115) can be directly related 
to a first-kind expansion in polynomials of odd degree, as follows. Write 
x = 2u? — 1, so that u = cos $8. We observe that, since (1.15) holds, namely 


Vi(x) = u-*Ton41(u), 


the third-kind expansion (5.112) gives 
f (Qu? — 1) ~yd Tosi (u (5.116) 


Thus, since the function f(2u? — 1) is an even function of u, so that the 
left-hand side of (5.116) is odd, the right-hand side must be the first-kind 
Chebyshev expansion of uf(2u? — 1), all of whose even-order terms must 
vanish. 
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Indeed, for the specific example 


f(x) =2-2(1—2)?, 


uf (2u? —1) =uV1—v? 


and hence we obtain the expansion 


V1— 2? ~S > ch Toi4i(2) (5.117) 


j=0 


we have 


where cY is given by (5.114). 


Fourth-kind expansions may be obtained in a similar way to third-kind 
expansions, simply by reversing the sign of wx. 


5.8.3 Multivariate Chebyshev series 


All the near-minimax results for first-, second-, third- and fourth-kind polyno- 
mials extend to multivariate functions on hypercubes, with the Lebesgue con- 
stant becoming a product of the component univariate Lebesgue constants— 
see Mason (1980, 1982) for details. 


5.9 Lacunary Chebyshev series 


A particularly interesting, if somewhat academic, type of Chebyshev series is 
a ‘lacunary’ series, in which non-zero terms occur progressively less often as 
the series develops. For example, the series 


Co 


= To(x) + $°(0.1)* 1" T5x (2) (5.118) 
k=0 


is such a series, since the degrees of the Chebyshev polynomials that occur 
grow as powers of 3. This particular series is also uniformly convergent, being 
absolutely bounded by the geometric progression 


Co 


So O.1)F = 2. 


k=0 


The series (5.118) has the remarkable property that its partial sum of 
degree N = 3”, namely 


Pn := To(x) + 5 (0.1)**1T34 (x), (5.119) 
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is a minimax approximation of degree (3"*! — 1) to f(x), since the error of 
this approximation is 


Co 


en = f(x) — pyle) = Y> OA) T(x), 


k=n+1 


and the equioscillating extrema of each of the polynomials T3. (#) for k > n+1 
include 3"*! + 1 extrema that coincide in position and sign with those of 
T3n+1(a); therefore their sum has equioscillating extrema at these same points, 
and we can apply the alternation theorem (Theorem 3.4). 


Generalising the above result, we can prove the following lemma and the- 
orem. 


Lemma 5.17 If r is an odd integer greater than 2, the polynomials T,.« (x), 
(k=n,n+1,...) have a common set of r” +1 extrema of equal (unit) mag- 
nitude and the same alternating signs at the points x = coskn/r", (k = 
0,1,...,7r”). 


Theorem 5.18 [fr is an odd integer greater than 2, and \\7- |ax| is con- 
vergent, then the minimax polynomial approximation of every degree between 
r” and r™t+ — 1 inclusive to the continuous function 


f(x) = S— axT,«(2) (5.120) 
k=0 
is given by the partial sum of degree r” of (5.120). 


A similar result to Theorem 5.18, for £; approximation by a lacunary 
series in U,«_ (x) subject to restrictions on r and ay, based on Theorem 6.10 
below, is given by Freilich & Mason (1971) and Mason (1984). 


5.10 Chebyshev series in the complex domain 


If the function f(z) is analytic within and on the elliptic contour E, (4.81) in 
the complex plane, which surrounds the real interval [—1, 1] and has the points 
z = +1 as its foci, then we may define alternative orthogonal expansions in 
Chebyshev polynomials, using the inner product (4.83) 


Canes a H2e@ lal |ael, (5.121) 


of Section 4.9 in place of (5.1). 
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Specifically, in the case of polynomials of the first kind, we can construct 
the expansion 


me / 
fe ~ S> ceTe(2) (5.122) 

k=0 

where (taking the value of the denominator from (4.85a)) 
(f , Te) 2 f aT |__o 
= = T, ; .123 
c (Ti, ; Tr) am (r2k ae r—2k) 2: f(z) K(Z) Te — pe (5 ) 
As in Section 4.9, we make the substitution (4.75) 

z=4t(wtw'), (5.124) 


under which the ellipse £,. in the z-plane is the image of the circle C’. of radius 
r > 1 in the w-plane: 


C, ={w:w= re? 6 real}. 


Then (4.77) 
and hence, for w on C,, 
Th(z) = 3 (w+ *) =F (r7 Fw tr we). (5.125) 


For w on C;., we also have 


= dd=—. (5.126) 


Define the function g such that for all w 


gw) = f(z) = fGwr+w™)); (5.127) 


then we note that g(w) will be analytic in the annulus between the circles C;, 
and C;,-1, and that we must have g(w~+) = g(w). 


Now we have 


a aaa | 22 
PT RCRD) i, F)T2) | a 
dw 
= ER f. g(w)(r?w-* + rw), (5.128) 


Since the function g(w) is analytic in the annulus between the circles C;, 
and C;-1, and satisfies g(w~') = g(w), we can show, by applying Cauchy’s 
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theorem to this annulus and then changing variable from w to w~', that 


dw dw 
¢ a(w)wk 2 = ¢ gw) = 
C:. 1W C,-1 1W 


Ck=— g(w)w" —. (5.130) 


fle) ~ ee § o(uyat 2h nce (5.181) 


or 


9 reff aout 2h ct (5.132) 


making use of (5.129) again. 


We may now observe that (5.132) is just the Laurent expansion of g(¢) in 
positive and negative powers of ¢. So, just as in the real case we were able 
to identify the Chebyshev series of the first kind with a Fourier series, in the 
complex case we can identify it with a Laurent series. 


5.10.1 Chebyshev—Padé approximations 


There is a huge literature on Padé approximants (Padé 1892)—rational func- 
tions whose power series expansions agree with those of a given function to as 
many terms as possible—mainly because these approximants often converge in 
regions beyond the radius of convergence of the power series. Comparatively 
little has been written (Gragg 1977, Chisholm & Common 1980, Trefethen & 
Gutknecht 1987, for a few examples) on analogous approximations by ratios of 
sums of Chebyshev polynomials. However, the Chebyshev—Padé approximant 
seems closely related to the traditional Padé table (Gragg & Johnson 1974), 
because it is most easily derived from the link to Laurent series via the prop- 
erty 
Tr(z) = 5(2e" +2”), 
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w and z being related by (5.124), so that we may match 


os 
s 
iv 
Sg 
om 
NI 
LON. 
§ 
cos 
| 
T 
i 
oo 
= 


co 
/ 
and y cha (w* + w*) 
L wk) k=0 


> 
iMs 
enh 
a 
= 
wle 
g 
= 
\ 


up to the term in w?t@+! 4 w-@t4a+) by multiplying through by the denom- 
inator and equating the coefficients of positive (or, equivalently, negative) and 
zero powers of w. 


There has also been work on derivations expressed entirely in terms of 
Chebyshev polynomials; the first that we are aware of is that of Maehly (1960) 
and a more efficient procedure, based on only p+ q+ 1 values of cx, is given 
by Clenshaw & Lord (1974). 


5.11 Problems for Chapter 5 


1. Verify the Chebyshev expansions of sgn x, || and d(x) quoted in (5.11), 
(5.12) and (5.24). 


2. If ¢; denotes the trapezium-rule approximation to c; defined by the right- 
hand side of (5.38), 2, being taken at the zeros of T;,(a), show that 


Cn = 0, 
Conti = —Ci, 
Can—i = C%- 


3. Show that the mapping S7, defined so that S7’f is the nth partial sum 
of the Chebyshev series expansion of f, is a projection. 


4. Prove (5.50): 


(a) directly; 


(b) by applying (1.14) and (1.15) to Exercise (3a) of Chapter 2 to 
deduce that 


and then making the substitution x7 = cos s. 


5. If A, is given by (5.71) show, using the inequality |sin st| < |5¢], that 


4 
An > —z logn. 
T 
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6. 


10. 


11. 


12. 
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With gp as defined by (5.82), show that if 7 is sufficiently small then 
the function gc defined by 


1 t+r 
go(t)'= 5 | go(s)ds 
T t—Tt 


has all the properties required to complete the proof in Section 5.5.1, 
namely that gc is continuous, even and periodic, ||gc||,, < 1+ and 
Igo — gpll, <e. 


. Assuming that f(z) is real when z is real, show that the coefficients c;, 


defined by (5.123) are the same as those defined by (5.7). 


. Consider the partial sum of degree n of the first kind Chebyshev series 


expansion of a function f(z), analytic on the interior of the ellipse EF, : 
|z + V2? —1| =r (r > 1) and continuous on E,. Show that this sum 
maps under z = $(w +w') into the partial sum of an even Laurent 
series expansion of the form oe cpw*, where c_~ = Ck. 


. Obtain Cauchy’s integral formula for the coefficients cy, and Dirichlet’s 


formula for the partial sum of the Laurent series, and interpret your 
results for a Chebyshev series. 


Following the lines of argument of Problems 8 and 9 above, derive partial 
sums of second kind Chebyshev series expansions of (z2—1)? f(z) and a 
related odd Laurent series expansion with c_, = —c,. Again determine 


integral formulae for the coefficients and partial sums. 


Using the Dirichlet formula of Problem 9, either for the Chebyshev 
series or for the related Laurent series, show that the partial sum is 
near-minimax on E,. within a relative distance ,,. 


Supposing that 


Gla) = a1 2) + VI a g(a) + 4/2 gs(x) +) S* gala), 


where 91, 92,93, 94 are continuously differentiable, and that 


n ; n 
g(x) ~ S- QarTy(t) ++++, ga(x) ~ Ss borUr—1(x) +++, 
r=0 r=1 


n-1 n-1 
g3(x) ~ Se G2ar41Vr(e) +++, ga(x) ~ a bar4iW;, (a) ++, 
r=0 r=0 


determine the form of F'(@) = G(cos @). Deduce that 


ilar 
F(20) = S© (ag cos kO + by sin kO) +++. 
k=0 
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Discuss the implications of this result in terms of separating a function 
into four component singular functions, each expanded in a different 
kind of Chebyshev series. 


CHAPTER 6 


Chebyshev Interpolation 


6.1 Polynomial interpolation 


One of the simplest ways of obtaining a polynomial approximation of degree 
n to a given continuous function f(x) on [—1, 1] is to interpolate between the 
values of f(x) at n+ 1 suitably selected distinct points in the interval. For 
example, to interpolate at 
U1,UQ,+++,En4+1 
by the polynomial 
Dn(x) = co Fea +--+ + en2", 


we require that 

Cot ate t-::++enep = fae) (K=1,...,n +1). (6.1) 
The equations (6.1) are a set of n+ 1 linear equations for the n+ 1 coefficients 
C0;-+-+;€n that define p,(z). 


Whatever the values of f(x,), the interpolating polynomial p,(2x) exists 
and is unique, since the determinant of the linear system (6.1) is non-zero. 
Specifically 


1 x x? ne 
1 2x ae a 
det = |[@ -=,) #0 
; ; i>J 
1 Sng. Fag 1° ny 


It is generally not only rather time-consuming, but also numerically unsta- 
ble, to determine p,,(a) by solving (6.1) as it stands, and indeed many more 
efficient and reliable formulae for interpolation have been devised. 


Some interpolation formulae are tailored to equally spaced points 21, 22, 

..5 n41, Such as those based on finite differences and bearing the names of 

Newton and Stirling (Atkinson 1989, for example). Surprisingly however, if 

we have a free choice of interpolation points, it is not necessarily a good idea 

to choose them equally spaced. An obvious equally-spaced set for the interval 
[—1,1] is given for each value of n by 


2k+1 


rane (k =0,...,n); (6.2) 


te=—-1+ 


these points are spaced a distance 2/(n + 1) apart, with half spacings of 
1/(n +1) between the first and last points and the end points of the interval. 
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(This set would provide equally spaced interpolation on (—oo, co) if f(x) were 
periodic with period 2.) However, the following example demonstrates that 
the points (6.2) are not appropriate for all continuous functions f(x) when n 
becomes large. 


Theorem 6.1 (Runge phenomenon) /f x, are chosen to be the points (6.2) 
for each n > 0, then the interpolating polynomial p,(x) does not converge uni- 
formly on [-1,1] as n — 00 for the function f(x) = 1/(1 + 2527). 


Figure 6.1: Interpolation to f(z) = 1/(1 + 25x?) by polynomials of degrees 
4 to 8 at evenly-spaced points (above) and at Chebyshev polynomial zeros 
(below) 


Proof: We refer the reader to (Mayers 1966) for a full discussion. The function 
f(z) has complex poles at z = +Hi, which are close to the relevant part of the 
real axis, and it emerges that such nearby poles are sufficient to prevent uniform 
convergence. In fact the error f(x) — pn(x) oscillates wildly close to x = +1, for 


large n. This is illustrated in the upper half of Figure 6.1. 


See also (Trefethen & Weideman 1991), where it is noted that Turetskii (1940) 
showed that the Lebesgue constant for interpolation at evenly-spaced points is 
asymptotically 2"t'/(en logn). @@ 


However, formulae are also available for unequally spaced interpolation, 
notably Neville’s divided-difference algorithm or Aitken’s algorithm (Atkinson 
1989) and the general formula of Lagrange quoted in Lemma 6.3 below. 
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A better choice of interpolation points to ensure uniform convergence, 
though still not necessarily for every continuous function, is the set of zeros 
of the Chebyshev polynomial T;,41(7), namely (as given in Section 2.2) 


(k-})n 


n+1 


XL = LE = COs (k=1,...,n+1). (6.3) 
This choice of points does in fact ensure convergence for the function of The- 
orem 6.1, and indeed for any continuous f(x) that satisfies a Dini—Lipschitz 
condition. Thus only a very slight restriction of f(x) is required. This is illus- 
trated in the lower half of Fig. 6.1. See Cheney (1966) or Mason (1982) for a 
proof of this. We note also from Theorem 6.5 that convergence in a weighted 
Lz norm occurs for any continuous f(z). 


By expressing the polynomial in terms of Chebyshev polynomials, this 
choice of interpolation points (6.3) can be made far more efficient and stable 
from a computational point of view than the equally-spaced set (6.2). So 
we gain not only from improved convergence but also from efficiency and 
reliability. We show this in Section 6.3. 


Finally, we shall find that we obtain a near-minimax approximation by in- 
terpolation at Chebyshev zeros, just as we could by truncating the Chebyshev 
series expansion — but in this case by a much simpler procedure. 


6.2. Orthogonal interpolation 


If {¢;} is any orthogonal polynomial system with ¢; of exact degree i then, 
rather than by going to the trouble of computing an orthogonal polynomial 
expansion (which requires us to evaluate the inner-product integrals (f , ¢;)), 
an easier way to form a polynomial approximation P,,(a) of degree n to a 
given function f(x) is by interpolating f(a) at the (n +1) zeros of bn41(2). 
In fact, the resulting approximation is often just as good. 


The following theorem establishes for general orthogonal polynomials what 
we already know in the case of Chebyshev polynomials, namely that ¢n41(2) 
does indeed have the required (n + 1) distinct zeros in the chosen interval. 


Theorem 6.2 If the system {¢;}, with 6; a polynomial of exact degree i, is 
orthogonal on [a,b] with respect to a non-negative weight w(x), then dn has 
exactly n distinct real zeros in [a,b], for every n> 0. 


Proof: (Snyder 1966, p.7, for example) Suppose that ¢, has fewer than n real 
zeros, or that some of its zeros coincide. Then there are m points f1, ta, ..., tm in 
[a, b], with 0 << m < n, where ¢,(x) changes sign. Let 

Im(x):=]][(e@-ti), m2>1; Mo(#):=1. 


i=l 
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Then II, is a polynomial of degree m <n, and so must be orthogonal to én. But 


(Im, bn) = / w(2) Tm ()bn(ae) dr # 0, 


since this integrand w(x)IIm(x)én(a) must have the same sign throughout the in- 
terval (except at the m points where it vanishes). We thus arrive at a contradic- 
tion. @@ 


Since an interpolant samples the values of the function in a discrete set 
of points only, it is usual to require the function to be in C[a, }] (i-e., to be 
continuous), even if we wish to measure the goodness of the approximation in 
a weaker norm such as Lo. 


Some basic facts regarding polynomial interpolation are given by the fol- 


lowing lemmas. 


Lemma 6.3 The polynomial of degree n interpolating the continuous function 
f(x) at then +1 distinct points 21, ..., 241 can be written as 


n+1 


Pn(x) = pa l;(a) f(a) (6.4) 


where ¢;(x) are the usual Lagrange polynomials 


ae) =] (=). (6.5) 


k=1 
hfi 
Lemma 6.4 /f x1, ..., Un41 are the zeros of the polynomial bn41(x), then 
the Lagrange polynomials (6.5) may be written in the form 
bn41(2) 


é(x) = (6.6) 


(@ — Bi) On 41 (wi) 


where ¢'(x) denotes the first derivative of (x). 


In the special case of the first-kind Chebyshev polynomials, the preceding 
lemma gives the following specific result. 


Corollary 6.4A For polynomial interpolation at the zeros of the Chebyshev 
polynomial T,,41(x), the Lagrange polynomials are 


Deis) 
Ae) Ge aoa) 
or 
cos(n + 1)@ sin 0; 
~ (n+ 1)(cos 6 — cos 6;) sin(n + 1)6; 
sin(n + 1)(@ — 6;) sin 6; 
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The following general result establishes £2 convergence in this framework 
of interpolation at orthogonal zeros. 


Theorem 6.5 (Erdés & Turdn 1987) If f(x) is in Cla,b], of {¢i(x), t = 
0,1,...} ts a system of polynomials (with d; of exact degree i) orthogonal 
with respect to w(x) on [a,b] and if py(x) interpolates f(x) in the zeros of 
gn+41(x), then 


b 
Jim (Uf —pa(o)ll)? = im, f w(2)(F@) — pn(a))? ae = 0. 


Proof: The proof is elegant and subtle, and a version for Chebyshev polynomials 
is given by Rivlin (1974). We give a sketched version. 


It is not difficult to show that {@;} are orthogonal. By ordering the factors 
appropriately, we can use (6.6) to write 


£;(x)e;(x) = dngi(z)bn—-1(z) (6435) 


where w,—-1 is a polynomial of degree n — 1. This must be orthogonal to ¢n+1 and 
hence 


(€i, €3) = (Ont, Yn-1) = 0. 
Therefore 
(€:, £))=0 (¢ #35). (6.8) 
Now 
If - polls < || ¢- 28 Dn — Pn 
where p? is the best £2 approximation. Therefore, in view of Theorem 4.2, it suffices 
to prove that 


+| 
2 


2 


= 0. 
2 


lim pB — Pn 


n—- co 


Since 
ntl1 


Dr (x) = Ss li(x)ph (wi) 


it follows from (6.4) and (6.8) that 


( 


Provided that (€;, €:) can be shown to be uniformly bounded for all i, the right- 
hand side of this equality tends to zero by Theorem 4.2. This certainly holds in the 


case of Chebyshev polynomials, where (¢;, £:) = * 7 ee 
n 


2 


B 
Pn — Pn 


a = ss (Li, &:) [f(ws) — pe (2s)] 


In the cases w(x) = (1+a)+2(1—«)*2, Theorem 6.5 gives £2 convergence 
properties of polynomial interpolation at Chebyshev polynomial zeros. For 
example, if x; are taken to be the zeros of T,41(2) then 

1 
lim | (1—2?)~2(f(x) — pp(x))? dx =0. 


no J_y 
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This result can be extended, and indeed Erdés & Feldheim (1936) have es- 
tablished £, convergence for all p > 1: 


1 
lim | (1—27)7?|f(x) —pa(z)|? dz =0. 


n>00 J_y 


In the case of Chebyshev zeros we are able to make more precise compar- 
isons with best approximations (see Section 6.5). 


If the function f(x) has an analytic extension into the complex plane, 
then it may be possible to use the calculus of residues to obtain the following 
further results. 


Lemma 6.6 If the function f(a) extends to a function f(z) of the complex 
variable z, which is analytic within a simple closed contour C that encloses 
the point x and all the zeros x1, ..., Un+41 of the polynomial bn41(x), then 
the polynomial of degree n interpolating f(x) at these zeros can be written as 


1 f (onsale) = dnl} f) 


Dn(x) = aa ESC) dz (6.9) 
and its error is 
1 f bun@F) a 
F(a) — ale) = 5p SE ae, (6.10) 


In particular, if f(a) extends to a function analytic within the elliptical 
contour FE, of Figure 1.5, then we can get a bound on the error of interpolation 
using the zeros of T;,41(x), implying uniform convergence in this case. 


Corollary 6.6A [If the contour C in Lemma 6.6 is the ellipse E,. of (1.84), 
the locus of the points }(re'® + r-te~®) as 6 varies (with r > 1, and if 
|f(z)| < M at every point z on E,, then for every real x on [—1, 1] we can show 
(see Problem 2) from (6.10), using (1.50) and the fact that |Tn41(x)| <1, that 


(r+r—1)M 
(ret — p—-n-1) (pr 4-1 — 2)’ 


lf (x) — pn(a)| < 


x real, -1<a< 1. 
(6.11) 


6.3. Chebyshev interpolation formulae 


We showed in Section 4.6 that the Chebyshev polynomials {T;(x)} of degrees 
up to n are orthogonal in a discrete sense on the set (6.3) of zeros {x;,} of 
Tn+1(x). Specifically 


ntl 0 ifj(<n) 


SUG Gi) = n+1 i=j7=0 (6.12) 
k=l (n+1) O<i=je<n. 
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This discrete orthogonality property leads us to a very efficient interpolation 
formula. Write the nth degree polynomial p,,(x), interpolating f(a) in the 
points (6.3), as a sum of Chebyshev polynomials in the form 


is / 
pray = ee) (6.13) 

i=0 
Theorem 6.7 The coefficients c; in (6.13) are given by the explicit formula 


2 n+1 


SF eT ae). (6.14) 
k=1 


OO ey 


Proof: If we set f(x) equal to pn(x) at the points {xx}, then it follows that 


n 


flee) = ST Tiler). 


i=0 


2 
Hence, multiplying by rama hi (xp) and summing, 
n 


= 6, 
from (6.12), giving the formula (6.14). @@ 
Corollary 6.7A Formula (6.14) is equivalent to a ‘discrete Fourier trans- 
form’ of the transformed function 
(0) = f(cos8). 


Proof: We have a 
Pn(cos 6) = ai cj. cos 70 
i=0 


with 
2 n+l 
ee » g(Ox) cos ix, (6.15) 
with 1) 
k—=)n 
6. = af 
n+1 20) 


Thus {c;} are discrete approximations to the true Fourier cosine series coefficients 


ce = = g(@) cos i0 dé, (6.17) 
a 


152 CHAPTER 6: CHEBYSHEV INTERPOLATION 


obtained by applying a trapezoidal quadrature rule to the (periodic) function g(6) 
with equal intervals 7/(n + 1) between the points 6%. Indeed, a trapezoidal rule 
approximation to (6.17), valid for any periodic function g(6), is 


n+1 1 ‘ 1 
1 k-—=5s uk —s 
Sui Sy (k— g)n eee Is 
qe es n+1 n+1 


which gives exactly the formula (6.15) for c; (when we note that the fact that both 
g(@) and cosi@ are even functions implies that the kth and (1 — k)th terms in the 


summation are identical), @®@ 


Thus, Chebyshev interpolation has precisely the same effect as taking the 
partial sum of an approximate Chebyshev series expansion, obtained by ap- 
proximating the integrals in the coefficients of the exact expansion by changing 
the independent variable from x to 6 and applying the trapezoidal rule — thus 
effectively replacing the Fourier transforms c? by discrete Fourier transforms 
c;. It is well known among practical mathematicians and engineers that the 
discrete Fourier transform is a very good substitute for the continuous Fourier 
transform for periodic functions, and this therefore suggests that Chebyshev 
interpolation should be a very good substitute for a (truncated) Chebyshev 
series expansion. 


In Sections 4.6.2 and 4.6.3 we obtained analogous discrete orthogonality 
properties to (6.12), based on the same abscissae x, (zeros of T,41) but 
weighted, for the second, third and fourth kind polynomials. However, it 
is more natural to interpolate a Chebyshev polynomial approximation at the 
zeros of a polynomial of the same kind, namely the zeros of Un41, Vn4i1, Wn+1 
in the case of second, third and fourth kind polynomials. We shall therefore 
show that analogous discrete orthogonality properties also follow for these 
new abscissae, and develop corresponding fast interpolation formulae. 


6.3.1 Aliasing 


We have already seen (Section 6.1) that polynomial interpolation at Cheb- 
yshev polynomial zeros is safer than polynomial interpolation at evenly dis- 
tributed points. Even the former, however, is unreliable if too small a number 
of points (and so too low a degree of polynomial) is used, in relation to the 
properties of the function being interpolated. 


One mathematical explanation of this remark, particularly as it applies to 


Chebyshev interpolation, is through the phenomenon of aliasing, described as 
follows. 


Suppose that we have a function f(x), having an expansion 


Co 
/ 


f@) = D) eiTi(2) (6.18) 
j=0 
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in Chebyshev polynomials, which is to be interpolated between its values at 
the zeros {x,} of Tn41(a) by the finite sum 


frlw) = S> &T;(2). (6.19) 
j=0 


The only information we can use, in order to perform such interpolation, 
is the set of values of each Chebyshev polynomial at the interpolation points. 
However, we have the following identity (where x = cos 6): 


T; (x) + Ton424; (2) = cos jO + cos(2n +2 + 7)0 


= + cos(n +1)0 cos(n+1+7)0 
— 4 Weie) Tn414;(2), (6.20) 


so that 


T; (tx) + Ton4o4+j(Lk) = 0, k= 1,...,n4+1. (6.21) 


Thus Ton+42+; is indistinguishable from —T; over the zeros of T,,+1. Figure 6.2 
illustrates this in the case n = 9, j = 4 (2n+2—j7 = 16). 


Figure 6.2: Tig(a) = —T4(a) at zeros of Ty9(x) 


In consequence, we can say that f,(x) as in (6.19) interpolates f(a) as in 
(6.18) between the zeros of T,,41(a) when 


Cj = Cj — Con42a—j — Con4o4j + Canga—jtCantaty—cc+, J =O0,...,n. (6.22) 


(Note that the coefficients Cn41, C3n43, ---do not figure in (6.22), as they 
correspond to terms in the expansion that vanish at every interpolation point.) 


In effect, the process of interpolation removes certain terms of the expan- 
sion (6.18) entirely, while replacing the Chebyshev polynomial in each term 
after that in T,,(x) by (41x) a Chebyshev polynomial (its ‘alias’) of lower 
degree. Since the coefficients {c;} tend rapidly to zero for well-behaved func- 
tions, the difference between c; and ¢; will therefore usually be small, but 
only if n is taken large enough for the function concerned. 


Aliasing can cause problems to the unwary, for instance in working with 
nonlinear equations. Suppose, for instance that one has a differential equation 
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involving f () and f(x)%, and one represents the (unknown) function f(a) in 
the form 7’ jo Cj 2; (x) as in (6.19). Then one might be tempted to collocate 
the equation at the zeros of T,,41(x) — effectively carrying out a polynomial 
interpolation between these points. Instances such as the following, however, 
cast doubt on the wisdom of this. 


In Figure 6.3 we have taken n = 4, and show the effect of interpolating 
the function T3(a)? at the zeros of T(x). (The expansion of f,(x)° includes 
other products of three Chebyshev polynomials, of course, but this term will 
suffice.) Clearly the interpolation is poor, the reason being that 


T3(x)? = + (To(a) + 3T3(x)) , 


which aliases to 
z (—Ti(x) + 3T3(2)). 


Figure 6.3: T3(x)° interpolated at zeros of Ts(2) 


In contrast, if we could have taken n = 9, we could have interpolated 
T3(x)° exactly as shown in Figure 6.4. However, we should then have had 


| NUZ 


Figure 6.4: T3(2)? interpolated (identically) at zeros of Tyo(a 


to consider the effect of aliasing on further products of polynomials of higher 
order, such as those illustrated in Figures 6.5 and 6.6. There are ways of 
circumventing such difficulties, which we shall not discuss here. 


Much use has been made of the concept of aliasing in estimating quadra- 
ture errors (see Section 8.4, where interpolation points and basis functions 
other than those discussed above are also considered). 


86.3: CHEBYSHEV INTERPOLATION FORMULAE 155 


Figure 6.6: T7(x)° interpolated at zeros of Tio(x) 


6.3.2 Second-kind interpolation 


Consider in this case interpolation by a weighted polynomial V1 — x? p,(2) 
on the zeros of U,+1(2), namely 


Yk = COS (kK=1,...,n+1). 


T 
n+2 
Theorem 6.8 The weighted interpolation polynomial to f(a) is given by 


1— 2? pr(a) = V1 — 2? > CU; (x) (6.23) 


with coefficients given by 


n+1 


qs dV! — UR F (yr)Ui(yn)- (6.24) 


4 


Proof: From (4.50), with n — 1 replaced by n + 1, 


n+1 : : . 
S- (1 = yR)Ui (ye )U5 (ye) = { 1( % t#I (Sn) (6.25) 


— n+1), t=j<n. 


If we set \/1 — yZpn(yx) equal to f(yx), we obtain 


= f1- Ad atin), 
i=0 
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and hence, multiplying by iV 1— y7U;(yx) and summing over k, 


n+1 n n+1 
= 7 d. 1— yg f(ye)U5 (Ye) = 28 I 20 - scidneicn)} 
=C 
by (6.25). @@ 


Alternatively, we may want to interpolate at the zeros of U,_1(x) together 
with the points x = +1, namely 


k 
Yk = COS — (k =0,...,7). 
n 


In this case, however, we must express the interpolating polynomial as a sum 
of first-kind polynomials, when we can use the discrete orthogonality formula 
(4.45) 


ae 0 BETIS 0); 
S* Tilye)Tj(ye) = 4 92, O< t= 5 <n; (6.26) 
k=0 n, t=j7=Ot=J7=n. 


(Note the double prime indicating that the first and last terms of the sum are 
to be halved.) 


The interpolating polynomial is then 


ie MN 
Pn(x) = S- iT; (x) (6.27) 
i=0 
with coefficients given by 
2 ” " 
ci=— D7 f(ve)Ti(ys). (6.28) 
k=0 


Apart from a factor of ,/2/n, these coefficients make up the discrete Cheby- 
shev transform of Section 4.7. 


6.3.3. Third- and fourth-kind interpolation 
Taking as interpolation points the zeros of V,41(2), namely 
(k=1,...,n+1), 


we have the orthogonality formula, for 7,7 <n, 


n+l 5 : 
Sa tewicawend={ 9.9 524 (6.29) 
k=1 7 


(See Problem 14 of Chapter 4.) 
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Theorem 6.9 The weighted interpolation polynomial to /1+ a f(x) is given 


by 
V1l4+a2pp(@)=V14+2 S¢ ciVi(a) (6.30) 
i=0 
where 
n+1 
= 5 > VI + teh (tK)Vilee). (6.31) 
ag 


Proof: If we set /1+ 2% pn(rer) equal to 1+ ref (xe), we obtain 


VT Fa flex) = VT Fan Yo aViles), 


i=0 
and hence, multiplying by - 5V1+ 2x V(x) and summing over k, 
aT? 
1 n+1 n 1 n+l 
x Dt 2k) f (ee) Vi (ee) = Saf : Su tavindiin)| 
N+ 5 hal i=0 Ho eet 
=o} 
by (6.29). @@ 


The same goes for interpolation at the zeros of W,+41(x), namely 


= phi 
Aaa ee 


2: 


LE = COS 


if we replace ‘V’ by ‘W’ and ‘1 +2’ by ‘1 — 2’ throughout. 


Alternatively, we may interpolate at the zeros of V,(a) together with one 
end point « = —1;i.e., at the points 


pa (k=1,...,n+1), 


where we have the discrete orthogonality formulae (the notation >” indicat- 
ing that the last term of the summation is to be halved) 


mH oo, eee) 
> Tien) T; (ee) = nth i=j= (6.32) 
k=1 s(n+4) O<i=je<n. 


The interpolating polynomial is then 


Pr(x) = } eT ;(z) (6.33) 
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with coefficients given by 


f(x)Ti(er). (6.34) 


6.3.4 Conditioning 


In practice, one of the main reasons for the use of a Chebyshev polynomial 
basis is the good conditioning that frequently results. A number of compar- 
isons have been made of the conditioning of calculations involving various 
polynomial bases, including {x*} and {T;,(x)}. A paper by Gautschi (1984) 
gives a particularly effective approach to this topic. 


If a Chebyshev basis is adopted, there are usually three gains: 


1. The coefficients generally decrease rapidly with the degree n of polyno- 
mial; 


2. The coefficients converge individually with n; 


3. The basis is well conditioned, so that methods such as collocation are 
well behaved numerically. 


6.4 Best £; approximation by Chebyshev interpolation 


Up to now, we have concentrated on the £. or minimax norm. However, 
the £,. norm is not the only norm for which Chebyshev polynomials can be 
shown to be minimal. Indeed, a minimality property holds, with a suitable 
weight function of the form (1—x)7(1+2)°, in the £; and Lz norms, and more 
generally in the £, norm, where p is equal to 1 or an even integer, and this is 
true for all four kinds of Chebyshev polynomials. Here we look at minimality 
in the £; norm. 


The £; norm (weighted by w(x)) of a function f(x) on [—1, 1] is 


ih= ff _w(e) [F(@)| de (6.35) 


and the Chebyshev polynomials have the following minimality properties in 
Ly. 


Theorem 6.10 2!~"T,,(x) (n> 0), 27"U,(x), 2-"Vp (x), 2-"W,, (x) are the 
monic polynomials of minimal Li norm with respect to the respective weight 
functions 


1 1 1 
1 


V1l—-22 ’ Vl—a’ Vita. 


w(x“) = (6.36) 
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Theorem 6.11 The polynomial pyn_1(x) of degree n—1 is a best Ly approzi- 
mation to a given continuous function f(x) with one of the four weights w(x) 
given by (6.86) if f(x) — pn—i(%) vanishes at the n zeros of Ty(x), Un(x), 
Vin(xz), Wr(x), respectively, and at no other interior points of [—1, 1]. 


(Note that the condition is sufficient but not necessary.) 


Clearly Theorem 6.10 is a special case of Theorem 6.11 (with f(x) = 2”), 
and so it suffices to prove the latter. We first state a classical lemma on the 
characterisation of best £1; approximations (Rice 1964, Section 4-4). 


Lemma 6.12 If f(x) — pn—1(x) does not vanish on a set of positive measure 
(e.g., over the whole of a finite subinterval), where pn—1 is a polynomial of 
degree n—1 in x, then py_1 is a best weighted Ly approximation to f on 
[—1,1] tf and only if 


HP = f w(a) sen{f(e)~pna(v)] on(z)de=0 (6.37) 


-1 


for r = 0,1,...,n—1, where each ¢,(x) is any given polynomial of exact 
degree r. 


Using this lemma, we can now establish the theorems. 


Proof: (of Theorem 6.11 and hence of Theorem 6.10) 


Clearly sgn(f (x) — pn—i(x)) = sgn Pn(x), where P, = T;, Uy, V>, Wr, respec- 
tively (r = 0,1,...,n). 


Then, taking ¢,(x) = P,(x) in (6.37) and making the usual change of variable, 


| sgn(cosn@) cos ré dé, 
0 


i sgn(sin(n + 1)@) sin(r + 1)6d0, 
I = Q 


sgn(cos(n + 4)0) cos(r + 4)0d9, 


Nie 


0, 


is sgn(sin(n 4 


0 


4)0) sin(r + 4)0d8, 


respectively. The proof that i = 0 is somewhat similar in each of these four cases. 


Consider the first case. Here, since the zeros of cosn@ occur at (k — 4)n/n for 
k=1,...,n, we have 


n/2n n-1 
I = / cosro.d0-+ 9 (-1)* | 
0) 


k=1 (k—4) n/n 


(k+5) n/n 


Tw 


cos r6 dé + (yr f cos r6 dé 


(n-5)n/n 


n-1 


II 
I 
D 
eB 
| 
+ 


(yh fain EE DE _ gy BoB 


k=1 
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_t 
4-11! gin $ 5)rn 
n 
2 r . orn eal (2n — 1)rm 
a5 es Si heh -1 
z sin 5 sin > + +(-1) n a 
1 2 -—1 
ae! [sin {ain 2 +sin 22} bee kC1y' win DEE [ot 
r n n n n 2n 


= 0. 


We can likewise show that I = 0 in each of the three remaining cases. Theo- 
rems 6.11 and 6.10 then follow very easily from Lemma 6.12 @@ 


It follows (replacing n by n+ 1) that the nth degree polynomial p,,() 
interpolating a function f(a) at the zeros of one of the Chebyshev polynomials 
Tn4i(2), Un+i(@), Vn4i(z) or Wr4i(x%), which we showed how to construct 
in Section 6.3, will in many cases give a best weighted £; approximation 
— subject only to the condition (which we cannot usually verify until after 
carrying out the interpolation) that f(2) —pp(x) vanishes nowhere else in the 
interval. 


6.5 Near-minimax approximation by Chebyshev interpolation 


Consider a continuous function f(«) and denote the (first-kind) Chebyshev 
interpolation mapping by J,. Then 


n+1 
(Inf \(e) = D7 Fon )en(2), (6.38) 
k=1 
by the Lagrange formula, and clearly J,, must be a projection, since (6.38) is 
linear in f and exact when f is a polynomial of degree n. From Lemma 5.13, 
J, is near-minimax within a relative distance ||Jn||,,- 


Now 5 
(Inf (@)| < So [NF lloo e(@)L- 
k=1 
Hence 
= sup ntlloe 
allen = SUP "TFT 
= Ir @)I 
OP eee aa ls 


n 


sup sup S- |é,(2)| 


f «é{-1,1] k—=0 


= bn (6.39) 


IA 
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where 
n+1 
Ln = sup So ex (a) (6.40) 
xe[-1,1] poy 


Now if 77-9 |\€x(x)| attains its extremum at x = €, we can define a con- 
tinuous function $(x) such that 


Ill. <4, 
b(xk) = sen(lx(E)). 


Then, from (6.38), 
n+1 


= Da lenlé )| = En, 


whence 
WFnlloo 2 llInPlloc = én: (6.41) 


Inequalities (6.39) and (6.41) together give us 
Il Jnlloo = 


What we have written so far applies to any Lagrange interpolation op- 
erator. If we specialise to first-kind Chebyshev interpolation, where ¢;() is 
as given by Corollary 6.4A, then we have the following asymptotic bound on 
IIJnlloo 


Theorem 6.13 [f {x} are the zeros of Tn41(x), then 


1. 
1 n+1 >. 4) 
In == >) EO (n+1)}? 
k=1 
2. 


2 
Ln = ~ log n + 0.9625 + O(1/n) as Nn — oo. 


Proof: For the details of the proof, the reader is referred to Powell (1967) or Rivlin 
(1974). See also Brutman (1978). @@ 


The following classical lemma then enables us to deduce convergence prop- 
erties. 


Lemma 6.14 (Jackson’s theorem) If w(65) is the modulus of continuity of 
f(x), then the minimax polynomial approximation B,,f of degree n to f sat- 
isfies 


If — Baflloo S w(/n). 
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Corollary 6.14A If (J, f)(x) interpolates f(x) in the zeros of Tn4i(x), and 
if f(a) is Dini—Lipschitz continuous, then (JInf)(a) converges uniformly to 
f(x) as n— oo. 


Proof: By the definition of Dini—Lipschitz continuity, w(d) log é6 > 0 as 6 — 0. By 
Theorem 5.12 


If -— Inf ll cc 


lA 


(1+ [Jnl oo If — Br fll 
< (1+ bn )w(1/n) 


2 
~ —w(1/n) logn 


= ~=.(6) logd (5 =1/n) 


— 0asd— 0; ie., asn — co, @@ 


In closing this chapter, we remind the reader that further interpolation 
results have been given earlier in Chapter 4 in the context of orthogonality. 
See in particular Sections 4.3.2 and 6.2. 


6.6 Problems for Chapter 6 


1. Prove Lemmas 6.3 and 6.4, and deduce Corollary 6.4A. 
2. Prove Corollary 6.6A. 


3. Find expressions for the coefficients (6.14) of the nth degree interpolat- 
ing polynomial when f(x) = sgna and f(a) = |x|, and compare these 
with the coefficients in the Chebyshev expansions (5.11) and (5.12). 


4. List the possibilities of aliasing in the following interpolation situations: 


(a) Polynomials U; of the second kind on the zeros of T;,+1(2), 
(b) Polynomials V; of the third kind on the zeros of T,,41(2), 

) Polynomials U; on the zeros of (1 — 2?)Un—1(2), 

(d) Polynomials T; on the zeros of (1 — x*)Un_1(2), 

(e) Polynomials V; on the zeros of (1 — 2?)U,_1(2), 


(f) Polynomials U; on the zeros of Un+1(2), 


(g) Polynomials T; on the zeros of U,,41(2). 


5. Give a proof of Theorem 6.11 for the case of the function U-,. 
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6. Using Theorem 6.11, consider the lacunary series partial sum 


n 


fal@) = S- CpUgn_1(2). 


k=1 


Assuming that the series is convergent to f = limyp—.oo fn, show that 
f — fn, for instance, vanishes at the zeros of Ugn_,. Give sufficient 
conditions for f, to be a best £; approximation to f for every n. 


7. Show that the n+ 1 zeros of Tn41(z) — Tn41(z*) are distinct and lie on 
E,, for a suitable fixed point z* on EF, (r > 1). Fixing r, find the zeros 
for the following choices of z*: 

a) zt = s(r+r-), 

b) z*=-3(rt+r74), 

(c) iger); 

) 


(d) z*= —4i(r —r}), 


R 
NIFR 


8. If fn(z) is a polynomial of degree n interpolating f(z), continuous on the 
ellipse FE, and analytic in its interior, find a set of interpolation points 
zp (kK =1,...,n+1) on E, such that 


(a) fy is near-minimax within a computable relative distance o, on 
E,, giving a formula for oy; 
(b) this result is valid as r — 1; i.e., as the ellipse collapses to the line 


segment [—1, 1]. 


To effect (b), show that it is necessary to choose the interpolation points 
asymmetrically across the z-axis, so that points do not coalesce. 


CHAPTER 7 


Near-Best £,,, £; and £, Approximations 


7.1 Near-best £,, (near-minimax) approximations 


We have already established in Section 5.5 that partial sums of first kind 
expansions 


(S2 f)(x -> CrUR (a =2 f Ane L(@)Te@) ee (7.1) 


yield near-minimax approximations within a relative distance of O(log) in 
C[-1, 1]. Is this also the case for other kinds of Chebyshev polynomial expan- 
sions? The answer is in the affirmative, if we go about the expansion in the 
right way. 


7.1.1 Second-kind expansions in L,, 


Consider the class C4;[—1,1] of functions continuous on [—1,1] but con- 


strained to vanish at +1. Let s®) f denote the partial sum of the expansion 
of f(x)/V1 — a? in Chebyshev polynomials of the second kind, {Uz(x) : k = 
0,1,2,...,n}, multiplied by V1 — «z?. Then 


(S@ f)(2) = V1—2? Yel bk = = [ f(@)Ug(x) dx. (7.2) 
WT SJ-1 


If now we define 
(0) = f(cos0) O<O0<7 
I) _F(cos0) —7 <0 <0 


(g(@) being an odd, continuous and 27-periodic function since f(1) = f(—1) = 
0), then we obtain the equivalent Fourier sine series partial sum 


nm 9) via 
(SPS, 9)(0) = S> by sin(k + 1)8, == | g() sin(k-+1)040. (7.3) 
k=0 0 


The operator $2’ fy can be identified as the restriction of the Fourier projection 
S, to the space CY, .. of continuous functions that are both periodic of period 


2m and odd; in fact we have SF°,g = SF 1g for odd functions g, where 
a sin(n + 3)t 

F 2 

6) = t dt. ‘ 
(S90) = sf ot+ 4 (7.4) 


165 
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If A, is the Lebesgue constant defined in (5.71) 


1 Tv 


sin(n + 4)t 
20 


An = 
aa 
sin st 


ae 


and partly tabulated in Table 5.1 on page 126, then, similarly to (5.75), we 
may show that 


|S |. = = ||5? Flies <An+1 (on the space C+;[—1, 1}). (7.5) 


Therefore (s?) f)(«) is near-minimax within a relative distance An41. 


This constant Aj+1 is not, however, the best possible, as has been shown 
by Mason & Elliott (1995) — the argument of Section 5.5.1 falls down because 


the function 
sin(n + 3)0 
. sin 40 


is even, and cannot therefore be closely approximated by any function in Coe 


However, g being odd, we may rewrite (7.4) as 


e sin(n + 3 
(S890) = Ef tole +0) - ot 9) EE” a 
oat sin(n + $)tt=8) sin(n + $Y(E+ 8) 
~ An e { sin $(t — 0) sind +(t + 0) \ . 
=z [ sKFso.0 an (7.6) 


This kernel K'5, (6, t) is an odd function of @ and t, and an argument similar 
to that in Section 5.5.1 can now be used to show that 


||| = Ist8i. = zesup ie IKFS,(6,t)| dt =, say. (7.7) 


Table 7.1: Lower bounds on 2) 
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Mason & Elliott (1995) have actually computed values of NP , which is no 
straightforward task since the points where the integrand kv a t) changes 
sign are not in general easily determined. For a lower bound to the supremum 
for each n, however, we may evaluate the integral when 0 = 1/(2n+3), when 
the sign changes occur at the precisely-known points t = 0, +37/(2n + 3), 
+57/(2n+ 3), ..., £7. This gives the values shown in Table 7.1. 


7.1.2. Third-kind expansions in L,, 


Following Mason & Elliott (1995) again, consider functions f in C_,[—1, 1], 
continuous on [—1,1] but constrained to vanish at x = —1. Then the nth 


degree projection operator Ss), such that s®) f is the partial sum of the 
expansion of f(x)\/2/(1 +.) in Chebyshev polynomials of the third kind, 
{Vi (a) :k =0,1,2,...,n}, multiplied by \/(1 + x)/2, is defined by 


(s® f\(a =F Deen (x) 


ore cos(k + 4 (7.8) 
k=0 


where x = cos@ and 


F(a) Vie 


otf) 


= -[° (0) cos(k + $)0d0 


T JO 
1 20 
=— g(9) cos(k + $)0.d0 (7.9) 
QT Qn 
with g defined as follows: 
f(cos0) O0<0<a7 
90) = 4 -g(2r-0) mS O< 20 


The function g() has been defined to be continuous (since g(7) = f(—1) = 0) 
and 47-periodic, and is even about 6 = y and odd about 0 = z. Its Fourier 
expansion Ca Teonemeui functions of $@) therefore involves only terms in 
cos(2k + 1)$ = cos(k + 4)@ and is of ihe form (7.8) when truncated. From 
(7.8) and (7. 9), 


(S f(z) = — / g(t) eos(k + B)teos(k + Haat 


20 
k=0 
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Table 7.2: Values of \°?) 


We leave it as an exercise to the reader (Problem 1) to deduce that 


(Sen) = 2 [e+ a, 


=e sin st 
Thus 
(SA@)] < Nall A (7.10) 
where 
1 [*|si Lt 
\@= -| ata dt. (7.11) 
T Jo sin xt 


Hence |s|| < v8), 


Arguing as in Section 5.5.1 as before, we again show that we have an 
equality 


| = 2), 


Numerical values of 2) are shown in Table 7.2, and clearly appear to 
approach those of A, (Table 5.1) asymptotically. 

A fuller discussion is given by Mason & Elliott (1995), where it is conjec- 
tured that (as for A, in (5.77)) 


4 
dS) = — logn + Ai + O(1/n) 


where A; ~ 1.2703. (This follows earlier work by Luttman & Rivlin (1965) 
and by Cheney & Price (1970) on the asymptotic behaviour of A,.) Once 
more, then, we have obtained a near-minimax approximation within a relative 
distance asymptotic to 477? log n. 


For further detailed discussion of Lebesgue functions and constants for 
interpolation, see Brutman (1997). 
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7.2  Near-best £; approximations 


From Section 6.4 we would expect Chebyshev series partial sums to yield near- 
best £1 approximations with respect to the weights given in (6.36), namely 
w(x) = 1/V1—22, 1, 1/Vl—<2a, 1/V1+2, since they already provide best 
£, approximations for a function that is a polynomial of one degree higher. In 
fact, this can be shown to hold simply by pre-multiplying and post-dividing 
the functions expanded in Section 7.1 by the additional factor V1 — «2. The 
simplest case to consider here is that of the second-kind polynomials U,,, since 
the function expanded is then just the original function. 


The partial sum of degree n of the second kind, for a continuous function 
f(x), is defined by the projection 


P®) : (P® f)(a =m by = ae V1 — x? f(x)Ug(x) dx. (7.12) 
TW SJ-1 


Defining the function g by 
g(0) = sin@ f (cos 8), (7.13) 
which is naturally an odd periodic continuous function, we see that 


a2 / * 9(6) sin(k + 1)8 a8, (7.14) 
0 


TT 


as in (7.3), and (P©) f)(cos0) = (S¥S,g)(0). 
Now, treating f(x) as defined on [—1,1] and g(6) as defined on [—7, 7] so 


that 


loll, a ia) a= [ ind f(cos6)| dd =2 f fle)dx=2 IF. 


we have 


PEF = F155, 
; in 1 
2 }_,|40 

Tv 1 Tw 
<3 [7 lool at Low [” [KES (6,0) a0 


= 4Flgll, A 
fll, AP, 


I 
I 


if g(t) K*?, (0, t) at do 


where \%) is the constant defined in (7.7) above. 
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Hence 
[P| <2), (7.15) 
1 


Thus x2) is a bound on | ge) 


P® | , Just as it was a bound on | 
1 


| in Sec- 
tion 7.1.1, and so (PY?) f) (x), given by (7.12), is a near-best £; approximation 
within the relative distance \) defined in (7.7). 

The discussion above is, we believe, novel. Freilich & Mason (1971) estab- 
lished that | Pp?) | is bounded by A», but the new bound (7.15) is smaller by 
about 0.27. : 


If we define PY and P®) to be the corresponding partial sum projections 
of the first and third kinds, 


Ch =— f(x)T, (x) da, (7.16) 


Ck = V2(1+ 2) f(x) Ve(x) da, (7.17) 


then it is straightforward to show in a similar way (see Problem 2) that 


PO) | <n (classical Lebesgue constant) 
1 


and 
[P| < 3) (given by (7.11). 
1 


7.3 Best and near-best £, approximations 


The minimal £,, and £; properties of the weighted Chebyshev polynomials, 
discussed in Sections 3.3 and 6.4, are in fact special examples of general Ly 
minimality properties, which are discussed by Mason & Elliott (1995). 


Theorem 7.1 The monic polynomials 2'~"T,,(z), 2~"Un(z), 2-"Vn(z), 
2-"W,,(z) minimise the Ly norm 


if w(x) |Pp(x)|? dx (l<p<o) (7.18) 
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over all monic polynomials P,,(x) with 
w(a) = (1—a)?(¢-Y(1 + 2) 206-Y 
for the respective values 


(a, 8) = (0,0), (Pp, P); (0, p), (p,0). 


The proof of this result depends on the characterisation of the best L, 
approximation according to the following result, which we state without proof. 


Lemma 7.2 The £L, norm (7.18) is minimised if and only if 


/ HOR Bb ede Oo: Sheu (7.19) 


-1 


Proof: (of Theorem 7.1) 


We shall concentrate on the first case, that of the first kind polynomials T, (x), 
and leave the remaining cases as exercises for the reader (Problem 3). 


Define 
P(x) =Tn(a), w(x) =1/./(1 — 2). 


Then 


1 wT 
| w(x) |Pr(a)|?~? Pr(x)Pe(x) da = / |cos n6|?~* cos n6 cos k6 dé. 
= 0 


at 


Now, for 0 < y < 1, define 


(|cos n8| < y), 


1 
Cr(O,y) = { 0 (\cosn6| > y). 


Then if y = cos7 we have C’,(0, y) = 1 over each range 


Thus, for any integer 7 with 0 < j < 2n, 


(rx—n)/n 
if cos 70 dé 
((r-1)r+n)/n 


n 


/ Cn(6, y) cos j0dd = 
0 


rel. 


“.1P. j{(r—1)0 . ara 
= So} fan AED tod _ 5, dior 


n n 
r=1 
2n P 
= >= sin H(r — 1) +n} 
r=1 J 1 


0. 
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But now, forO<k<n, 


1 
[ w(o) Palo)? Pa) P(x) an 
-1 
= | \cos n6|?~? cosné cos ké dé 

0 
= | |cos n6|?~? 3[cos(n + k)@ + cos(n — k)6| dé 


= | {4 a O04) ay} 1 [cos(n + k)0 + cos(n — k)6] 40 


peace i y?} {/ (1-—Cn(9,y)) 4{cos(n + k)@ + cos(n — k)6] ao} dy 


The result then follows from Lemma 7.2. @® 


(An alternative method of proof is to translate into polynomial terms 
the result on trigonometric polynomials, due to S. N. Bernstein, given in 
Achieser’s book (Achieser 1956, Section 10).) 


7.3.1 Complex variable results for elliptic-type regions 


It is possible to obtain bounds for norms of projections, and hence measures 
of near-best £, approximation, by using ideas of convexity over a family of £, 
measure spaces for 1 < p < oo (Mason 1983), Mason 1983a). However, the 
settings for which there are results have been restricted to ones involving gen- 
eralised complex Chebyshev series — based on results for Laurent series. Ma- 
son & Chalmers (1984) give £, results for Fourier, Taylor and Laurent series; 
moreover Chalmers & Mason (1984) show these to be minimal projections on 
appropriate analytic function spaces. The settings, involving projection from 
space X to space Y, where A(D) denotes the space of functions analytic in 
D and continuous on D, are: 


1. Chebyshev, first kind: X = A(D,), where D, is the elliptical domain 
128 |z + V22-— 1| < p}; Y = Y; =I, (polynomials of degree n in z); 


P=G, 


where G,, is the Chebyshev first-kind series projection of A(D,) into 
II,. 


2. Chebyshev, second kind: X = {f(z) = Vz?-—1F(z), F € A(D,)}. 
Y=Y¥o={f(z) =vV2?2-1F(z), Fe Ty}; 


P=H* _,:H* ,f=V2-1n-iF, 
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where H,, is the Chebyshev second kind series projection of A(D,) into 
Il,. 


3. Generalised Chebyshev: X = A({z: pi < |z+Vz?—1| < p2}) (annu- 
lus between two ellipses); Y = Yi ® Yo; 


P=Jn=Gn+H*_,. 


Then it is proved by Mason (19830), using convexity arguments, that for each 
of the three projections above 


Pll, < (oan)? “1 (1 < p< ov) (7.20) 
where 
1 f" |si 
on = ~ | ane! a ao. 
n Jo sin 6 


Note that oo, = An. So the generalised expansion is proved to be as close to 
minimax as the (separated) first kind one. 


For p = 1, p = o, we obtain bounds increasing as 477? logn, while 
|Pll, > 1 as p— 2. 


It follows also (Chalmers & Mason 1984) that J, is a minimal projection; 
indeed, this appears to be the only such result for Chebyshev series. The 
component projections G,, and H;_, are essentially odd and even respectively, 
and correspond to the cosine and sine parts of a full Fourier series. In contrast, 
the projection G,, is not minimal. 


The earliest near-best results for £4. and £; approximation on elliptic 
domains appear to be those of Geddes (1978) and Mason (1978). See also 
Mason & Elliott (1993) for detailed results for all individual cases. 


We should also note that it has long been known that 
IPll, < Cp (7.21) 
where C, is some constant independent of n. Although this is superficially 
stronger than (7.20) from a theoretical point of view, the bounds (7.20) are 
certainly small for values of n up to around 500. Moreover, it is known that 


Cy > cas p— oo. See Zygmund (1959) for an early derivation of this result, 
and Mhaskar & Pai (2000) for a recent discussion. 


7.4 Problems for Chapter 7 


1. Show that 


cos(k + $)tcos(k + $)0 = $[cos(k + $)(t + 0) + cos(k + $)(t — )| 
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and that 
nm * 1 
S- cos(k i 1)y = alt = yu 
k=0 sin zu 
Hence prove that 
ae “ 4 : 1 a sin(n + 1)t 
_ ; s k+5)0d0= — t+ 6)————_ dt 
=f, Loeosth + Hteos +5)0d =a) g(t + 8) ne d 


by showing that the pair of integrals involved are equal. 


(This completes the proof of Section 7.1.2, showing that the weighted 
third-kind expansion has a partial sum which is near-minimax.) 


2. Show that | 


Pal | anand |p | < ane where A, is the classical 
1 1 
Lebesgue constant and v2) is given by (7.11). 


3. Prove Theorem 7.1 in the case of polynomials of the second and third 
kinds. 


4. If S,, is a partial sum of a Fourier series 


n 


(Sn f)(0) = $40 + S— (ax cos kO + by, sin k8), 
k=0 


show how this may be written, for suitably defined functions, as a com- 
bined first-kind and (weighted) second-kind Chebyshev expansion. 
(Hint: f(0) = F(cos 6) + sin@ G(cos 0) = even part of f + odd part of 


5. Consider the Fejér operator F,,, which takes the mean of the first n 
partial sums of the Fourier series. 


(a) Show that F;, is not a projection. 
(b) Show that 


where 
_ sin $(n+1)6 sin $n0 


5,(0 

ee) 2sin 50 

(c) Show that (F;,f)(@), under the transformation x = cos 6, becomes 
a combined third-kind and fourth-kind Chebyshev-Fejér sum, each 
part being appropriately weighted. 


6. Derive the basic result for p = oo, namely ||P||,, < can = An, for the 
three projections listed in Section 7.3.1. 
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10. 


11. 
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. Derive the corresponding basic results for p = 1. 


Would it be possible to obtain a better set of results in this case by 
using an odd kernel, like that used in (7.6)? 


. Note that ||P||, = 1 in Section 7.3.1 and that it is known that ||P||,, is 


bounded for any fixed p in the range 1 < p < oo. Discuss whether there 
is a ‘better’ result than the one quoted. 


(You might like to consider both the practical case n < 500 and the 
theoretical case of arbitrarily large n.) 


. Investigate the validity of letting p — 1 in the results of Section 7.3.1, 


when the interior of the ellipse collapses to the interval [—1, 1]. 


ral in the case n = 0. 


| Co 


Compute by hand the bounds for 


Compute some numerical values of ?) and compare them with the 
lower bounds given in Table 7.1. 


CHAPTER 8 


Integration Using Chebyshev Polynomials 


In this chapter we show how Chebyshev polynomials and some of their funda- 
mental properties can be made to play an important part in two key techniques 
of numerical integration. 


e Gaussian quadrature estimates an integral by combining values of the 
integrand at zeros of orthogonal polynomials. We consider the special 
case of Gauss—Chebyshev quadrature, where particularly simple proce- 
dures follow for suitably weighted integrands. 


e One can approximately integrate a function by expanding it in a series 
and then integrating a partial sum of the series. We show that, for 
Chebyshev expansions, this process — essentially the Clenshaw-Curtis 
method — is readily analysed and again provides a natural procedure 
for appropriately weighted integrands. 


Although this could be viewed as an ‘applications’ chapter, which in an 


introductory sense it certainly is, our aim here is primarily to derive further 
basic properties of Chebyshev polynomials. 


8.1 Indefinite integration with Chebyshev series 


If we wish to approximate the indefinite integral 


where —1 < X < 1, it may be possible to do so by approximating f(x) on 
[—1, 1] by an nth degree polynomial f,,(x) and integrating w(x) fn (x) between 
—land X, giving the approximation 


x 
A(X) & hy (X) =| w(x) fn(x) da. (8.1) 
-1 
Suppose, in particular, that the weight w() is one of the four functions 
1 1 1 
w(2) = ——=, 1 (8.2) 


Vil—-a22 ? Jl—a’ Vita’ 


and that we take f,(a) as the partial sum of the expansion of f(x) in Cheb- 
yshev polynomials of the corresponding one of the four kinds 


P,, (x) = T, (x), Uz (2), Vi. (2), W,(2). (8.3) 


177 
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Then we can use the fact that (excluding the case where Py (a) = T;,(a) with 
k =0) 


x 
/ w(t) Pe (2) de = O(X)Qx(X) — Cx(—1)Oe(-) 


-1 


where 
Qk (X) => Up-1(X), Tr4i(X), W(X), V(X) (8.4a) 
a v1 — X? VI—X VI+X 
1-— x? 1 1-X 14+ X 
= = Ab 
Cr(X) eee ee (8.4b) 


respectively. (Note that C,(—1) = 0 in the first and fourth cases.) This 
follows immediately from the fact that if « = cos@ then we have 


d k cos k@ 
— sink? = — 
dx Se sing ’ 
d (k + 1)sin(k + 1)6 
— .] k 1 0 = ekea—_—_————— 
dz coset) sin 0 : 
k+ 4)cos(k + 4)0 
4 antes dy — Seta) cosh + 20 
dx Bath 2) sin 0 : 
d i (k + 5) sin(k + 4)0 
ae OE NO an 
In the excluded case, we use 
d 1 
tay see 
dx sin 0 
to give 
a 4 
———. 7 (x) dx = arccos(—1) — arccos X = 7m — arccos X. 
| aro (-1) 


Thus, for each of the weight functions (8.2) we are able to integrate the 
weighted polynomial and obtain the approximation h,,(X) explicitly. Suppose 
that 


fr(e) = S~ axTe(a) [Pe =Te] or S>anPe(a) [Pe =Us, Vis We) (8.5) 
k=0 k=0 


Then in the first case 
n , xX 
hn(X) = So ag / w(a)Ty (x) dx = 
k=0 = 


“ Vl- xX? 
= $a0(m — arccos X) — dae Up-1(X), (8.6) 
k=1 
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while in the second, third and fourth cases 
n x n = 
ES a | w(2)Pp(2) dt = Doan (Ce(@Qe(a)%,- (8-7) 
k=0 = k=0 


The above procedure is a very reliable one, as the following theorem 
demonstrates. 


Theorem 8.1 If f(x) is £L2-integrable with respect to one of the weights w(x), 
as defined by (8.2), and hy(X) is defined by (8.6) or (8.7) as appropriate, if 
Qi(X) and Cy(X) are defined by (8.4), and if ax are the exact coefficients 
of the expansion of f(x) in Chebyshev polynomials of the corresponding kind, 
then hn(X) converges uniformly to h(X) on [1,1]. 


Proof: The idea of the proof is the same in all four cases. We give details of the 
second case here, and leave the others as exercises (Problems 1 and 2). 


For Py = Ux, w = 1, 
xi 
is fn(x) dx 


[ Sewsine + yoae, 


1 K=0 


hn(X) 


Thus the integrand is the partial Fourier sine series expansion of sin 0 f(cos @), which 
converges in £2 and hence in £; (Theorems 5.2 and 5.5). 


is (f(a) — fa a) } de 


if Ife) = fo(a)| de 


Now 


[|b — Arll go 


mans 


A 
5 
© 
* 


|f(@) — fn(x)| da 


I 
io 
a B 


sin 6 f (cos @) — Dios k+1)0| do 


ef 


— 0, n-o. 
Hence hy converges uniformly toh. @®@ 


The coefficients a, in (8.5) have been assumed to be exactly equal to the 
relevant Chebyshev series coefficients. In practice, we most often approximate 
these by the corresponding coefficients in a Chebyshev interpolation polyno- 
mial (see Chapter 6) — effectively evaluating the integral that defines a, by 
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the trapezoidal rule (see Section 6.2). In some circumstances, we may need 
to calculate the Chebyshev coefficients more accurately than this. 


The method followed above is equivalent to methods well known in the 
literature. For the first choice (P;,, = Tj) the method is that of Clenshaw & 
Curtis (1960) and for the second choice (P;,, = U;,) that of Filippi (1964). 


The analysis of Section 8.1 is taken mainly from Mason & Elliott (1995, 
and related papers). 


8.2 Gauss—Chebyshev quadrature 


Suppose that we now wish to calculate a definite integral of f(x) with weight 
w(x), namely 


b 
f=} f(x) w(a) dz. (8.8) 
Suppose also that J is to be approximated in the form 
I~ S> An f (xr) (8.9) 
k=1 


where A, are certain coefficients and {x,} are certain abscissae in [a, b] (all 
to be determined). The idea of Gauss quadrature is to find that formula (8.9) 
that gives an exact result for all polynomials of as high a degree as possible. 


If Ini f(a) is the polynomial of degree n — 1 which interpolates f(x) in 
any n distinct points 71, ..., Yn, then 


Taf) = > fee) (8.10) 
k=1 


where ¢; is the Lagrange polynomial (as in (6.5)) 


é,(x) = I (==) (8.11) 


r£k 


The polynomial J,_1 f(x) has the integral 


b 
In, =f In-1f (a) w(a) da 
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provided that the coefficients A, are chosen to be 


b 
A= [ w(a)ep (a) da. (8.12) 


With any n distinct abscissae, therefore, and with this choice (8.12) of coef- 
ficients, the formula (8.9) certainly gives an exact result whenever f(x) is a 
polynomial of degree n — 1 or less. We can improve on this degree, however, 
by a suitable choice of abscissae. 

Notice too that, for general abscissae, there is no control over the signs 
and magnitudes of the coefficients Aj, so that evaluation of the formula (8.9) 
may involve heavy cancellation between large terms of opposite signs, and 
consequent large rounding error. When we choose the abscissae to maximise 
the degree of exactness, however, it can be shown that this problem ceases to 
arise. 


Theorem 8.2 If x, (k =1,...,n) are the n zeros of ¢n(x), and {op : k = 
0,1,2,...} 2s the system of polynomials, dy having the exact degree k, orthog- 
onal with respect to w(x) on [a,b], then (8.9) with coefficients (8.12) gives an 
exact result whenever f(x) is a polynomial of degree 2n—1 or less. Moreover, 
all the coefficients A, are positive in this case. 


Proof: Since ¢n(x) is a polynomial exactly of degree n, any polynomial f(x) of 
degree 2n — 1 can be written (using long division by ¢n) in the form 


f(&) = bn(#)Q(a) + In-1 f(x) 


where Q(x) and Jn—1 f(x) are polynomials each of degree at most n — 1. Then 
b b b 
i, f(x)w(ax) dx =) on(x)Q(x)w(x) ac [ Ini f (@) w(x) da. (8.13) 


Now ¢n(x) is orthogonal to all polynomials of degree less than n, so that the 
first integral on the right-hand side of (8.13) vanishes. Thus 


[ feeyw(oyaa = fi Ini f (x) w(x) dx 


= S> Ak Jn—if (we) 


k=1 


since the coefficients have been chosen to give an exact result for polynomials of 
degree less than n. But now 


f(&x) = On(te)Q(ee) + In-1f (€e) = In-1f (re), 


since x, is a zero of ¢n(ax). Hence 


ff foywa) de =F Anson), 
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and so (8.9) gives an exact result for f(a), as required. 


To show that the coefficients A, are positive, we need only notice that ¢, (x)? is 
a polynomial of degree 2n — 2, and is therefore integrated exactly, so that 


n b 
An = >> Ajlx(aj)” = | lx (x)? w(x) dx > 0 
j=l e 


for each k. @®@ 


Thus we can expect to obtain very accurate integrals with the formula 
(8.9), and the formula should be numerically stable. 


When the interval [a,b] is [—1,1] and the orthogonal polynomials ¢,,(x) 
are one of the four kinds of Chebyshev polynomials, then the weight function 
w(x) is (1 — a?)~2, (1— 2?)?, (1+2)2(1—2)~? or (1— 2)32(1+2)-? and 
the zeros x, are known explicitly. It remains to determine the coefficients Ax, 
which we may do by making use of the following lemma. 


Lemma 8.3 


cos 8 — cos¢ sing ’ 


f cos nd ao sinnd 
0 


* sinné sind 
f eee ee cos ng, 
9 cosé@—cos¢@ 


for any ¢ in [0,7], n = 1,2,3,.... 


(We have stated this lemma in terms of the ‘Cauchy principal value’ in- 
tegral f--- dO since, if we allow ¢ to take an arbitrary value, the integrands 
have a non-integrable singularity at 6 = ¢. However, when we come to apply 
the lemma in this chapter, 9 = ¢ will always turn out to be a zero of the 
numerator, so that the singularity will in fact be removable and the principal 
value integrals will be equivalent to integrals in the ordinary sense.) 


Proof: The lemma can be proved by induction on n, provided that we first establish 
the n = 0 case of the first result 


1 
———_ dé =0. 
f cos # — cos@ 


We may do this as follows. Since cos @ is an even function, we have 


i 1 
—_- 4g = 
{ cos 8 — cos @ 
1 
=i ————__ dé 
af cos 0 — cos d 


_ ie ei? dé 
~ J (e% — &4)(e9 — e-18) 
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_ f —idz 
= |z|=1 (z — el#)(z — e718) 


oe i dz dz 
ee ee | fie, Bee jel 2 — *? 


oe Oe: 
= rer ae =0. 


We leave the subsequent induction as an exercise (Problem 3). @@ 


The evaluation of A; can now be carried out. 


Theorem 8.4 In the Gauss—Chebyshev formula 


a f(x)w(x) da ~ So Ax f (ae), (8.14) 


k=1 


where {xz} are the n zeros of bn(x), the coefficients Ax are as follows: 


1. For w(x) = (1 —2?)~2, bn(x) = Ta(2): 


2. For w(x) = (1—22)?, bn(x) = Un(x): 


Proof: We prove case 1 and leave case 2 as an exercise (Problem 4). We shall 
prove cases 3 and 4 a little later. 


In case 1, writing 


k—4)r 
Lp = COS Op ee eet 
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for the zeros of T,,(x), 


_ f Te) de 
ve ie (a — &p) NUn-1(ae) VI — a? 


® cosné sin 0; 
= ———___________—*__—__ dg 
9 (cosé —cosOx) n sin nO,% 


using Corollary 6.4A and Lemma 8.3. @®@ 


Case 1 above is particularly convenient to use, since all weights are equal 
and the formula (8.9) can thus be evaluated with just n —1 additions and one 
multiplication. 


EXAMPLE 8.1: To illustrate the exactness of (8.9) for polynomials of degree < 2n—1, 
consider n = 4 and f(x) = 2”. Then 


Ta(x) = 8x* — 827 +1 


has zeros 71,...,2%4 with 
2 2 2+V2 2 2_ 2-V2 
t= = mi 2 = %3 >= ml ‘ 
Hence ; : 
= dxw Sap = 52 2+ v2 2-v2 ae. 
-_1 V1 — 22 4 3 4 4 4 2 


which is the exact value of the integral, as we expect. (See Problem 6 for a more 


challenging example.) 


Cases 3 and 4 of Theorem 8.4, namely the Chebyshev polynomials of the 
third and fourth kinds, require a little more care. We first establish a lemma 
corresponding to Lemma 8.3. 


Lemma 8.5 


m +4)0 in(n+ 4 
f cos(n + 5) eos pag 7 sin(n 2)? 
0 


cos 6 — cos¢@ a sin $¢ 


f sulci Hae 1949 = _meos(n + 3)6 
0 


cos 0 — cos @ 2 sindg 
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Proof: (of the Lemma) From the first equation of Lemma 8.3, if we replace cos 0 
by x and cos ¢ by y, 


ae ee 
f. €=y4 fla = TUn-1(y). (8.15) 


Writing « = 2u? — 1, y = 2v? — 1, where u = cos 40, v = cos 44, 
1 3 1 
f Ge) Va (x) ac = f 2u Lea) aa; 
ag Vee xr-y o Vl—w we—v 
1 
1 1 du 
Tan ———— 
f tenn (Stas) ase 
t Ton+1(u) du 


~ fy, u-v fl—-w 
= TU2n(v), — by (8.15). 


Ne 


Rewriting this in terms of @ and ¢, we get 


ia +4)0 in(2n + 1)4 
f a COEF aes gdp ee ae (8.16) 
9 sin5@ cos@—cos¢ sin $4 


and this proves part 1 of the Lemma. 


Part 2 may be proved similarly, starting from the second equation of Lemma 8.3, 
which gives 


and making similar substitutions. @@ 


Proof: (of Theorem 8.4, case 3) Here 
t (1 +2 2 L—- 2x 
Ay = ——_)d 
. EG) (=) * 


) 
_ f 1 cos(n + 3)9 cos $0, sin O, sin 
0 


—— — OOo do 
sin 40 (cos 6 — cos 6%) (n+ 4) sin(n + 4)0; 
= ci cos” 40%, by (8.16) 
nT 9 
= —*_(1+2,). 
n+3 


Thus case 3 is proved. Case 4 follows, on replacing x by —xz. @®@ 
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EXAMPLE 8.2: To illustrate this case, consider, for example, f(x) = x? andn=2 
for case 3, so that 


Now V(x) = 4x? — 22 — 1 has zeros 21, 22 = +(1 + V5), with «7, 23 = 3(3 + V5). 
Hence 


(a+ v1) 2} +(1+ r2)x3] 


= —[$(5+ V5)4(3 + V5) + $(5 — V5)4(3 — V5)] 


This is exact, as we can verify: 


™ cos 40 7 
Fs i) <2" (cos6)” sind dd = if 4(1 + cos6)(1 + cos 26) d0 = dr. 
9 sin 359 F 


The Gauss—Chebyshev quadrature formulae are the only Gauss formulae 
whose nodes x, and weights A; (given by Theorem 8.4) can be written down 
explicitly. 


8.3. Quadrature methods of Clenshaw—Curtis type 


8.3.1 Introduction 


The Gauss—Chebyshev quadrature method of Section 8.2 is based on the con- 
tinuous orthogonality properties of the Chebyshev polynomials. However, as 
we showed in Section 4.6, the four kinds of polynomials also have discrete 
orthogonality properties, and it is this kind of property that was exploited in 
the original quadrature method of Clenshaw & Curtis (1960). Their method 
has been developed in a considerable literature of papers by many authors 
(Piessens & Branders 1983, Adam 1987, Adam & Nobile 1991); a particularly 
nice presentation is given by Sloan & Smith (1978), who provide a version 
based on a general weight function together with a calculation of error esti- 
mates. Our treatment here is based on Sloan and Smith’s formulation and 
techniques, which we can extend to all four kinds of Chebyshev polynomials. 


The basic idea is to replace the integrand by an interpolating polynomial, 
and then to integrate this between the required limits. Suppose that we wish 
to determine the integral 


I(f) = / wo(a) f (2) de: (8.17) 
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then we replace f(x) by the polynomial J,, f(a) of degree n which interpolates 
f in abscissae {x, :k =1,...,n +1}, and hence we obtain the approximation 


1 


TIn(f) =) w(x) Inf (x) dx (8.18) 


-1 


to evaluate, either exactly or approximately. So far, this only repeats what 
we have said earlier. However, if Chebyshev polynomial abscissae are adopted 
as interpolation points then, as we saw in Section 6.3, discrete orthogonal- 
ity properties lead to very economical interpolation formulae, expressing the 
polynomial J, f(a) in forms which can readily be integrated — in many cases 
exactly. 


There are a few important cases in which Gauss—Chebyshev and Clenshaw-— 
Curtis quadrature lead to the same formulae, although they differ in general. 


8.3.2 First-kind formulae 
Suppose that 


Inf (2) = } > b;T;(a) (8.19) 
j=0 


interpolates f(x) in the zeros {x,} of Tn4i1(a). Then, using the discrete 
orthogonality results (4.40) and (4.42), we have 


n+1 
i= > TaphG yao 143, agen (8.20a) 
k=1 
and 
__ J +1), i=9, 
dis = { Lin+1), iO (8.20b) 
Hence 
n+1 n+1 n n+l 
Yo flee) Tien) = D0 Inf (we) Ti(we) = D0; S) Taw) Ty (we) = dich 
k=1 k=1 j=0 k=l 
and so 
1 n+1 
b= = So f (we) Tian). (8.21) 
4 k=1 
From (8.18) 


Ef) =>. byaj, (8.22) 
j=0 
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where : 
ay =| w(x)T; (x) da =} w(cos 6) cos j@ sind dé. (8.23) 
-1 0 
Formulae (8.21)—(8.23) give the quadrature rule 
n+1 
In(f) = >> wef (ae), (8.24a) 
k=1 
: aj “/ 2a; 
=) = ~ T; (xk). (8.24b) 
j=0 diy oe ! 


Hence I, is readily determined, provided that the integrals (8.23) defining a, 
are straightforward to calculate. 


e For the specific weighting 


we have 


1 wT 3 
_ _ 2\-$ 7. _ ee a, 20, 
a; = i (1 — 2°)" 2T;(x) dx = | cos 79d = { ee eer (8.26) 


1 


giving 
ao Tv 
fp —T =— — _. 
ae (rE) aia 


Hence 
n+1 


TT 
In ——_——— . 2 
[ toBeenn=h df). 20 
Thus we get the first-kind Gauss—Chebyshev formula of Theorem 8.4. 


An alternative Clenshaw—Curtis formula may be obtained by defining 
Jnf (x) to be the polynomial interpolating the values of f(z) at the abscissae 
k 
YR = Cos —, k=0,...,n, 
n 


which are the zeros of (1 — 2?)Un—i(x). In this case we use the discrete 
orthogonality results (4.45) and (4.46) to give us 


dij := S~ Tilye)Tj(yx) =0, 145 (8.28a) 


and 
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We readily deduce, in place of (8.19), that 


Inf (a) = >) bjT;(2) (8.29) 
j=0 
where in this case 7 
1 ” 
b= dD flun)Ti(ur), (8.30) 
ah k=0 


and that 


where a; are given by the same formula (8.23) as before. This gives us the 
rule 


Tn(f) = S- wef (ye) (8.31a) 
k=0 

we = >> = Ti(yn) = s~ “41 Ty(yp) (8.31b) 
j=0 49 j=0 


: dz T 
| tose =r =2 "Few. (8.32) 


This is nearly equivalent to the second-kind Gauss—Chebyshev formula 


of Theorem 8.4, applied to the function oy except that account 


is taken of the values of f(a) at the end points « = +1. This may 
better reflect the inverse-square-root singularities of the integrand at 
these points. 


8.3.3. Second-kind formulae 


It is clear that the key to the development of a Clenshaw—Curtis integration 
method is the finding of a discrete orthogonality formula. In fact, there exist 
at least sixteen such formulae, listed in Problem 14 of Chapter 4, some of 
which are covered in Section 4.6. 


An example of a second-kind discrete orthogonality formula, given by 
(4.50) and (4.51), is 


n+l 1 a a _ 
dij = Sot — yi )Ui(yR)U; (ye) = { 2( rey a2; 7, (8.33) 
k=1 ’ 
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where {y,} are the zeros of Un41(2): 


kr 


—., k=1,...,n+1. 
n+2 


Yk = COS 


To make use of this, we again approximate the required integral I(f) of 
(8.17) by the integral [,,(f) of the form (8.18), but now interpolating f(a) by 


a function of the form 


Inf (x) = (1— 27)? S° 6,0; (a); (8.34) 
j=0 


that is, a polynomial weighted by (1 — 22)?. There is thus an implicit as- 
sumption that f(#) vanishes at x = +1, and that it possibly has a square-root 
singularity at these points (though this is not essential). 


Now 


2 n+l r 
bj = — y?)2 : 
aa aS Yk)? F (ye Ui(yn) (8.35) 
from (8.33). Integrating (8.18) gives us 
F,(f) = 7 bya; (8.36) 
j=0 
where 


a; = a w(x)(1 — x?)2U; (2x) dz = ie w(cos 0) sin(j +1)0 sin@dé. (8.37) 


This gives the rule 


In(f) = So wef (yn); (8.38a) 
k=1 
wr = (1 yp)? Ss? aes U3; (Yk) (8.38b) 
j=0 


e In the special case where w(x) = 1, 


1 us 1 
_ see Ary. = : 2 z _ BY = 
a; -| (1 —2*)2U;(x) dx : sin(j +1)6 sin@ dé { oe i. 


-1 


Hence, from (8.36), (8.37), 


f(x) de = In(f) = > S192)? Fyn). (8.39) 
k= 
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This is equivalent to the second-kind Gauss—Chebyshev formula of The- 


orem 8.4, applied to the function 
f(«) 


JI — x2 


8.3.4 Third-kind formulae 
A third-kind formula is obtained from the orthogonality formula 


n+3, i=j, 
0, i# Jj, 


(8.40) 


n+1 


k=1 
where {x} are the zeros of V,41(x). (See Problem 14 of Chapter 4.) 


In this case, we choose 
n 


a polynomial weighted by (1 + x)? (implicitly supposing that f(—1) 


Now, from (8.39), we can show that 
n+1 : 
(1+ rx)? f(x) Vilar). 


b; = 
3 
n+ > k=1 


Integrating (8.18) gives us again 
In(f) = > bya 
j=0 
(8.43) 


where now : 
a= | und nTey as 


So we have the rule 
(8.44a) 


k=1 
(8.44b) 


(8.45) 
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then 
1 
da 
a; = | (1+2)V;(e)——— = 
s= f Oto 
= . Gi 1 ui =) T, 7=0, 
-{ 2eos(j-+ $)0 cos 30.a0 = { SO 
Hence 


: dx on oe 1 
[ FOF RAM Fag Lote). 640) 


This is equivalent to the third-kind Gauss—Chebyshev formula of Theo- 
rem 8.4, applied to the function 


f(x) 
lta 


8.3.5 General remark on methods of Clenshaw—Curtis type 


There are effectively two types of quadrature formula considered above. 


e For special choices of weight function w(x), such that all but one of 
the Chebyshev transforms b; vanish, the formula involves only a single 
summation — such as (8.27) — and is identical or very similar to a 
Gauss—Chebyshev formula. 


e For a more general weight function, provided that the integral (8.23), 
(8.37) or (8.43) defining a; can be exactly evaluated by some means, we 
obtain a formula involving a double summation — such as (8.24) — one 
set of summations to compute the weights wz and a final summation to 
evaluate the integral. 


8.4 Error estimation for Clenshaw—Curtis methods 


There are a number of papers on error estimation in Clenshaw—Curtis methods 
(Fraser & Wilson 1966, O’Hara & Smith 1968, Smith 1982, Favati et al. 1993, 
for instance). However, we emphasise here the approach of Sloan & Smith 
(1980), which seems to be particularly robust, depends on interesting proper- 
ties of Chebyshev polynomials, and is readily extendible to cover all four kinds 
of Chebyshev polynomial and the plethora of abscissae that were discussed in 
Section 8.3. 
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8.4.1 First-kind polynomials 
Suppose that the function f(x) being approximated is continuous and of 


bounded variation, and therefore has a uniformly convergent first-kind Cheb- 
yshev expansion 


fla) & S> Bj Tj(e). (8.47) 
j=0 


Then the error in the integration method (8.31) (based on {yx}) is 


= (T-Ih)| 35 B72) 


jg=nt+l1 
= SO Bi AU(T;) -— In(Tj)}. (8.48) 
g=nt+l 
Now : 
1(T,) = i wath de = a (8.49) 
-1 
and (J, again denoting the operator interpolating in the points {y;}) 
1 
In(Tj) =| w(x)InT; (x) da. (8.50) 
-1 
But 
InT (Ye) = Tj (Ye) = Tyr (ye) (8.51) 


where (as shown in Table 8.1) 7’ = 7’(m, 7) is an integer in the range 0 < j’<n 
defined by 


J(n,j) = 3b, O<jen 
j'(nm,j) = In-j, n<jeran >. (8.52) 
j'(n,2n+j) = j'(n,J) 


This follows immediately from the observation that, 7, k and n being integers, 


(2n + j)kr 


jkr 
T; (yx) = cos — = cos = Tontj (Yr). 


Thus the interpolation operator J, has the so-called aliasing: effect of 
identifying any Chebyshev polynomial 7; with a polynomial 77, of degree at 
most n, and it follows from (8.51) that, identically, 

InT;(x) = Ty (2), (8.53) 


1See Section 6.3.1. 
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Table 8.1: T;:(x) interpolates T;(x) in the zeros of (1 — x?)Un—1(z) 


j= | 0 1 2 > n-lin 
2n|2n—1 2n-2 « n+1 n 


and 
In(T5) = In(Ty) = LD) = a5". (8.54) 
Therefore a 
En(f) = S> Bj(aj — 457). (8.55) 
g=nt+l 


Sloan & Smith (1980) assume that the weight function w(x) is smooth 
enough for a; (8.23) to be neglected for 7 > 2n and that the integrand f(z) 
itself is smooth enough for (; (8.47) to be neglected beyond j = 3n. Then 
(8.55) yields, referring to Table 8.1, 


|En(f)| < |an41 =. an—1| Gn+1| ah lant2 _ An—2| |Bn+2| apes 
+++ dan — ao| |Gan| + lar| |Gen4i] +--+ + |an| [Gan] - 


If we then assume a geometric decay in the (js, say 


|Bn+5| as Cait, 
for some Cp, Tn With rp, < 1, then 
IEn(f)| < en{l@nti — @n—1] Pn ++ +++ |@en — aol 7 +] a1| ne “+ |an| ile 
(8.56) 


If we change the notation slightly, replacing b; by bn;, the additional sub- 
script being introduced to show the dependence on n, 


n 
a 


bng == 32" Fw Ti(on), 


k=0 


it is clear that b,; is an approximation to 


Deir daz 
B= Ff tenet 


which becomes increasingly accurate with increasing n. Hence, a succession 
of values of 6,,; (for various values of n) may be used to estimate 6;. (For the 
case j =n, 3; would be approximated by $bn,;-) 
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Sloan and Smith’s ‘second method’ is based on obtaining estimates of r, 
and cp, and then using them in (8.56). Essentially, r,, is estimated from ratios 
of coefficients and c, from the coefficients themselves. One algorithm, which 
takes account of the observed fact that odd and even coefficients tend to have 
somewhat different behaviours, and which uses three or four coefficients to 
construct each estimate, is as follows: 


e Compute 


41 = max{5 lbnn| , |bn,n—2l 5 |On,n—4| 5 |bnjn—ol}, 


22> max{|bnn—1| ’ lon n—3| ’ [Brees | . 
e If 21 > z then if |bnn—6| > ++: > 4 |bnn| then 


5 lbnnl [Pnin—21 [Pr.n—a 
r= max | oP intel As ar \ ' 8.57 
[Bn n—2| |bnn—4| lbnn—6| ( ) 


otherwise r, = 1. 
e If z1 < zg then if |byn—5| > +--+ > |bnn—1i| then 
2 |bnn—1| eel 
r, = max, ——_, ——_ >, 8.58 
{as |bnn—5| ( ) 
otherwise r, = 1. 


e Set 
= max{4 lBnn|; |Pnn—il Tn, ---> [On n—6l ro}. (8.59) 


8.4.2 Fitting an exponential curve 


A similar but somewhat neater procedure for estimating c, and ry is to fit 
the coefficients 
Ban; bnn—1; bnn—2; nas »Onn—k 


(or the even or odd subsequences of them) by the sequence 
n 
Cats Gilee SCAT Gate Cnl 


This is in effect a discrete approximation of a function g(x) = bax by 


Siar Ses 


at r=n,n—1,n—2,...,n—k, where A=Inc, and B= Inry. 


Then 
g(x) = eA +7" + e(zx) 
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where e(x) is the error. Hence 
Ing(x) + In(1 — e(x)/g(x)) = A+ Ba 
so that, to the first order of approximation, 
In g(x) — e(x)/g(x) = A+ Ba 


and 


g(x) In g(a) — e(a) © g(a)(A+ Ba). 


Hence a discrete least-squares fit of In g(a) by A+ Ba, weighted throughout 
by g(a), can be expected to give a good model of the least-squares fitting of 


g(x) by e4t+ Be, 


This is an example of an algorithm for approximation by a ‘function of a 
linear form’ — more general discussion of such algorithms is given in Mason 


& Upton (1989). 


8.4.3 Other abscissae and polynomials 


Analogous procedures to those of Section 8.4.1 can be found for all four kinds 
of Chebyshev polynomials, and for all sets of abscissae that provide discrete 


orthogonality. 


For example: 


e For first-kind polynomials on the zeros {x,} of Tn+41(«) (8.24), equations 


(8.47)-(8.50) still hold, but now 


InT;(k) = Tj (ae) = £Tj (ve) 


where (as in Table 8.2) 


Jj) = §, 0 <j <n (with + sign) 
J ree n+1) = n+l (with zero coefficient) 
j'(n,j) = In+2—-j, n+2<j<2n+2 (— sign) 
j(n,j+2n+2) = j'(n,7) (with changed sign) 
(8.60) 
This follows immediately from 
cos oa 2 (O<j<n) 
0 (jg =n+1) 
T; a n el\ ee ee 
(x) op Ce ee a TMK ) = — cos Hg (n+2<7<2n+4 2) 
eres =-cos4 (erg) (Qn+3 <j <3n+2) 
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Table 8.2: +T;/(x) interpolating T;(x) in the zeros of T,,41(x) 


n+2 —4 2nd 
+3 | 38n+2 <— 2n4 
+6 — 6n4 


We now deduce that 


IEn(f)| < langa| /Bntil + |@n+2 + @n| |Bn+al +++ 
-+ + |d@on42 + Go| |Gon+2| + |ar| [Genta] +--- 
“++ |an4i| [Pan+al - (8.61) 


For second-kind polynomials on the zeros of U,+1 (8.38), we require an 
expansion 


f(x) = S— B;U;(2) 
j=0 


so that 3, is approximated by 6; from (8.35). 
Then 


En(f)= S > (Uj) — In(Us)] 


j=nt+l 
where now 
Key if. (2) 2)? Us @y de = ws (8.62) 
and 
AG) = / _w(e)(1 2°)! J,05 (0) de. (8.63) 


If {y,} are the zeros of Un+i(x), then 


In U5 (Ye) = U5 (Ye) = EU; (ye) 
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where (taking U_; = 0) 


J (nj) = §, 0<j <n (with + sign) 
j ie n+1) = n+l1 (with zero coefficient) 
j'(n,j) = In+2—-j, n+2<j<2n+2 (— sign) 
j'(n,2n+3) = -1 (with zero coefficient) 
J(n,j+2n+4) = j'(n,j) (with unchanged sign) 
(8.64) 
This is shown in Table 8.3, and follows from 
kar 
Yk = COS Of CON Tg ptuct, Wee 
For 
U; (ye) sind, = sn(j+1)6, (9 =0,...,n) 
= sin(2n + 2— 7’ +1)6, = —sin(j’ + 1)6% 
= —U;' (yx) sin Ox (j/ =n+1,...) 
and 


sin(j + 2n + 4+ 1), =sin(j + 1)0,. 


Table 8.3: +U;/(x) interpolating U;(x) in the zeros of U,;+1(x) 


From (8.62) and (8.63): 


> Bj (a; 


g=nt+l 
and 
[En(f)| < langal[Bnti] + l@nte + Gn] |Bn+2l + °°: 


*+ + [dante + ao| |Gon+2| + lao] |Gonyal + °° 
*+ + |an4i| [3n-+5] - (8.65) 


88.4: ERROR ESTIMATION FOR CLENSHAW-CURTIS METHODS 199 


e For third-kind polynomials on the zeros of V;,41 (8.44), we use an ex- 


pansion 
[QRH Uae BV): (8.66) 
j=0 
Then - 
Ea(f)= >> BlE(V;) - (V5) 
j=nt+l1 

where F 

1¥j) =f w(a)(d +2)'?Vj(2) de = a, (8.67) 
and , 

I,,(V;) =i w(x)(1 + 2)/? JnV; (a) dx. (8.68) 


Choose {2} as the zeros of V,,41(x). Then 


InVj(a) = Vj (ek) = Vy (ee) 


where 
i(nj) = 3, 0 <j <n (with + sign) 
j(innt+1) = n+l (with zero coefficient) 
ji(n,j) = M+2—j, nt$2<j <$In+2(— sign) 
j(n,j+2n4+3) = 7’(n,J) (with changed sign) 
(8.69) 
This is shown in Table 8.4, and follows from 
k—i 
Lp = cos Oy = Cos ee 1g 
2: 
giving 
j4+1)(p—4 
cos 40,V; (an) = ge as 
(Qn+2—j'+ 5)(k—3)a 
a 
gg (20+ Y= G+ Yk = He 
n+ 3 
(j' + gk 3)a 
= — cos 5 
and 
og GHIN+S+HVMK= Hm  MK— Be 
n+ 3 n+ 3 
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Table 8.4: +V;/ (x) interpolating V;(x) in the zeros of V,,+1(z) 


n+2 —> 2n+2 
t-4)3n+3 <— 2n+3 


+8 — 6n4 


From (8.67) and (8.68): 
En(f) = S> Bj(aj — 457) 
jg=nt+l 
and 
IEn(f)| < langal [Gn+il + l@nte + @n| [Pn+2| +> 
+ + |dan42 + ao] |B2n| + lao] |Gan43] + °° 
-+ + |@n| |G3n+al - (8.70) 


We note that there are only very slight differences between Tables 8.2, 8.3 
and 8.4 and between the corresponding error bounds (8.61), (8.65) and (8.70). 


8.5 Some other work on Clenshaw—Curtis methods 


There is now a significant amount of literature on Clenshaw—Curtis methods, 
built up over about forty years, from which we shall draw attention to a 
selection of items. 


Of particular interest are applications to Bessel function integrals (Piessens 
& Branders 1983), oscillatory integrals (Adam 1987), Fourier transforms of 
singular functions (Piessens & Branders 1992), Cauchy principal-value inte- 
grals (Hasegawa & Torii 1991) and Volterra integral equations (Evans et al. 
1981). 


Among contributions specific to error bounds and error estimates are the 
early work of Chawla (1968), Locher (1969) and O’Hara & Smith (1968), 
together with more recent work of Smith (1982) and Favati et al. (1993)—the 
last being concerned with analytic functions. 


Product integration (including error estimation) has been well studied, in 
particular by Sloan & Smith (1978, 1980, 1982) and Smith & Paget (1992). 


There has been an important extension of the Clenshaw—Curtis method 
to integration over a d-dimensional hypercube, by Novak & Ritter (1996). 
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Problems for Chapter 8 


If w = (1—2?)~? and Fy(x) = T;(a) in Section 8.1, show that 


dx 


I~ alee = f 0-2") -4 
= 


By considering the Fourier cosine series expansion of f(cos@), deduce 
Theorem 8.1 for the first case. 


-> apTe (x 


f (cos 0) — or cos k0} dé. 


k=0 


.Ifw=[s(1- x)|~2 and Py(x) = Vz(a) in Section 8.1, show that 


|b — Palle 


> OnVE (a 


1 
f go-or 
=] 
°f 
0 
n/2 
| 
0 


By considering the Fourier cosine series expansion of cos@ f(cos2¢) 
(which is odd about ¢ = $7), deduce Theorem 8.1 for the third case. 


I 


cos $8 f (cos @) — Si apoos (k+4 


cos ¢ f(cos 2) — 3 ax cos(2k + 1)¢ 


k=0 


d¢. 


. Complete the proof of Lemma 8.3, by performing an induction on n for 


the pair of formulae together. 


. Use Lemma 8.3 to prove the second part of Theorem 8.4. Verify that 


this quadrature formula is exact for n = 3 in the case of the integral 


1 
/ 1 — 22 27 de. 
= 


. Prove in detail the second part of Lemma 8.5. 


. Verify the exactness of Gauss-Chebyshev quadrature using first-kind 


polynomials, by testing it for n =4 and f(x) = 2°, f(x) =a" 


. Verify the Gauss—Chebyshev rule for fourth-kind polynomials, by testing 


it forn = 1 and f(z) = 1, f(x) = 


. Verify that there is a Gauss—Chebyshev quadrature rule based on the 


zeros of (1 — x?)U,_-1(x) and the polynomials T;,(z), and derive a for- 
mula. (This type of formula, which uses both end points, is called a 
Lobatto rule.) When would this rule be useful? 
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9. Show that there is a Gauss-Chebyshev quadrature rule based on the 
zeros of (1+ 2)V,(a#) and the polynomials T(x), and derive a formula. 
(This type of formula, which uses one end point, is called a Radau rule.) 
When would this rule be useful? 


CHAPTER 9 


Solution of Integral Equations 


9.1 Introduction 


In this chapter we shall discuss the application of Chebyshev polynomial tech- 
niques to the solution of Fredholm (linear) integral equations, which are clas- 
sified into three kinds taking the following generic forms: 


First kind: Given functions K(x,y) and g(x), find a function f(y) on [a, 6] 
such that for all x € [a, bj 


b 
‘ K (x,y) f(y) dy = 9(2); (9.1) 


Second kind: Given functions K(x, y) and g(x), and a constant A not pro- 
viding a solution of (9.3) below, find a function f(y) on [a, 6] such that 
for all x € [a, 0] 


b 
f(a)-d / K(,v)f(w) dy = g(a); (9.2) 


Third kind: Given a function K(z, y), find values (eigenvalues) of the con- 
stant \ for which there exists a function (eigenfunction) f(y), not van- 
ishing identically on [a, 6], such that for all x € [a, }] 


b 
fle)—a ‘ K (x,y) f(y) dy =0. (9.3) 


Equations of these three kinds may be written in more abstract terms as 
the functional equations 


Kf =4q, (9.4) 
f-AKF = 9, (9.5) 
f—-AKf = 0, (9.6) 


where K represents a linear mapping (here an integral transformation) from 
some function space F’ into itself or possibly (for an equation of the first kind) 
into another function space G, g represents a given element of F or G as 
appropriate and f is an element of F' to be found. 


A detailed account of the theory of integral equations is beyond the scope 
of this book — we refer the reader to Tricomi (1957), for instance. However, 
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it is broadly true (for most of the kernel functions K(x, y) that one is likely to 
meet) that equations of the second and third kinds have well-defined and well- 
behaved solutions. Equations of the first kind are quite another matter—here 
the problem will very often be ill-posed mathematically in the sense of either 
having no solution, having infinitely many solutions, or having a solution f 
that is infinitely sensitive to variations in the function g. It is essential that 
one reformulates such a problem as a well-posed one by some means, before 
attempting a numerical solution. 


In passing, we should also mention integral equations of Volterra type, 
which are similar in form to Fredholm equations but with the additional prop- 
erty that K(x,y) = 0 for y > x, so that 


b 
i K (x,y) f(y) dy 


is effectively 
| K(2,y) f(y) dy. 


A Volterra equation of the first kind may often be transformed into one of the 
second kind by differentiation. Thus 


[Kent ay = (2) 


becomes, on differentiating with respect to 2, 


K(ee)fa) + [EK ef) dy = Lalo). 


It is therefore unlikely to suffer from the ill-posedness shown by general Fred- 
holm equations of the first kind. We do not propose to discuss the solution of 
Volterra equations any further here. 


9.2 Fredholm equations of the second kind 


In a very early paper, Elliott (1961) studied the use of Chebyshev polynomials 
for solving non-singular equations of the second kind 


b 
f(a) -d / K(e,9fu)dy=9(2), a<e<s, (9.7) 


and this work was later updated by him (Elliott 1979). Here K(z,y) is 
bounded in a < x,y < b, and we suppose that A is not an eigenvalue of 
(9.3). (If A were such an eigenvalue, corresponding to the eigenfunction ¢(y), 
then any solution f(y) of (9.7) would give rise to a multiplicity of solutions of 
the form f(y) + ad(y) where a is an arbitrary constant.) 
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For simplicity, suppose that a = —1 and b= 1. Assume that f(x) may be 
approximated by a finite sum of the form 


x iN 
ye a;T;(x). (9.8) 


j=0 


Then we can substitute (9.8) into (9.7) so that the latter becomes the approx- 
imate equation 


N N i 
A A 

So" ajTi(x)—¥ Soa; | K(w,y)Tjly) dy ~ gla), —-1 <2 <1. (99) 

j=0 j=0 


We need to choose the coefficients a; so that (9.9) is satisfied as well as 
possible over the interval —1 < « <1. A reasonably good way of achieving 
this is by collocation — requiring equation (9.9) to be an exact equality at 
the N +1 points (the extrema of Ty(x) on the interval) 


ease oh on 
X= Yi,n = COS W’ 
so that 
ou W 
Y= 45(Pig — AQiz) = Gyn), 1=0,...,N, (9.10) 
j=0 
where : 
Py=Tilvin)s y= f E(u w)D(u) ae (9.11) 


We thus have N + 1 linear equations to solve for the N + 1 unknowns a;. 


As an alternative to collocation, we may choose the coefficients b; , so that 
Kw(yi,n,Y) gives a least squares or minimax approximation to K(y:,Nn, y)- 


If we cannot evaluate the integrals in (9.11) exactly, we may do so approx- 
imately, for instance, by replacing each K(y;,n,y) with a polynomial 


Ku(yinsy) = >_ bixTe(y) (9.12) 
k=0 


for some! M > 0, with Chebyshev coefficients given by 
bik — cag K(yi,.n;Ym,M)Tk(Ym,M); k= 03 .,M, (9.13) 


m=0 


1There does not need to be any connection between the values of M and N. 
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where 
MT 
Ym,M = COS Fs m=0,...,M. 


As in the latter part of Section 6.3.2, we can then show that 


Ku (Yin, ¥m.M) = K (yin, Ym,M), m=0,...,M, 


so that, for each 7, Kys(yi,n,y) is the polynomial of degree M in y, interpo- 
lating K(yi,n,y) at the points Ym,m.- 


From (2.43) it is easily shown that 


1 —2 
/ T,(t) de=< 2-1? "°° (9.14) 
-1 0, n odd. 
Hence 
1 
Qi; y+ Ku (Yin; y)T; (y) dy 


lI 
NIK 
S 
> 
Uae 
A 
oa 
+ 
> 
Yate. 
ce 
es 
te 
aa 
+ 
= 
Fae: 
cS 
eases 
SH 
Q 
Keg 


II 
| 
BS 
ca 
—“——s 
grt: 
&. 
+ 
=o 
N 
| 
fan 
— 
GS 
| 
=o 
eo 
| 
an 
Sa 


jtk even 
u ptk—-1 
=_2 bi. -——— 9.15 
> "GP ke = 1)? = apr? iad 


giving us the approximate integrals we need. 


Another interesting approach, based on ‘alternating polynomials’ (whose 
equal extrema occur among the given data points), is given by Brutman 
(1993). It leads to a solution in the form of a sum of Chebyshev polyno- 
mials, with error estimates. 


9.3. Fredholm equations of the third kind 


We can attack integral equations of the third kind in exactly the same way as 
equations of the second kind, with the difference that we have g(x) = 0. 
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Thus the linear equations (9.10) become 


N 
”" 


a;(Pi; — AQiy) = 0, 7=0,...,N. (9.16) 
j=0 
Multiplying each equation by Ty(yi,) and carrying out a >” summation 
over i (halving the first and last terms), we obtain (after approximating K by 
Ky) the equations 


M NM 7 
" " 
Bde os 


Tu(vin YK. thm. )Ti(Um.as) f Telu)Ts(v) dy J 
k=0 i=0 m=0 =A 
(9.17) 
which is of the form se 
MN 
ra =r) Ag (9.18) 
j=0 
or, written in terms of vectors and matrices, 
MN 
t= Aa. (9.19) 


Once the elements of the (N+1) x (N+1) matrix A have been calculated, this 
is a straightforward (unsymmetric) matrix eigenvalue problem, which may 
be solved by standard techniques to give approximations to the dominant 
eigenvalues of the integral equation. 


9.4 Fredholm equations of the first kind 


Consider now a Fredholm integral equation of the first kind, of the form 


b 
g(a) = / ACHR PEC e ae (9.20) 


We can describe the function g(a) as an integral transform of the function 
f(y), and we are effectively trying to solve the ‘inverse problem’ of determining 


f(y) given g(x). 
For certain special kernels K (x,y), a great deal is known. In particular, 
the choices 


K(a,y) =cosay, K(a#,y) =sinzy and K(z,y) =e ", 
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with [a, b] = [0, co), correspond respectively to the well-known Fourier cosine 
transform, Fourier sine transform and Laplace transform. We shall not pursue 
these topics specifically here, but refer the reader to the relevant literature 
(Erdélyi et al. 1954, for example). 


Smooth kernels in general will often lead to inverse problems that are 
ill-posed in one way or another. 


For example, if K is continuous and f is integrable, then it can be shown 
that g = Kf must be continuous — consequently, if we are given a g that is 
not continuous then no (integrable) solution f of (9.20) exists. 


Uniqueness is another important question to be considered. For example 
Groetsch (1984) notes that the equation 


| xsiny f(y) dy=a 
0 


has a solution 


However, it has an infinity of further solutions, including 
f(y)=3tsinny (n=2,3,...). 


An example of a third kind of ill-posedness, given by Bennell (1996), is 
based on the fact that, if K is absolutely integrable in y for each x, then by 
the Riemann—Lebesgue theorem, 


b 
On(2) =| K(a,y) cosny dy > 0 as n > ov. 


Hence 


b 
J Kev) Fu) + acosny) dy = of) + a6n(2) + gle) a8 n— 00, 
a 
where a is an arbitrary positive constant. Thus a small perturbation 


dg(x) = adbn(x) 


in g(x), converging to a zero limit as n — oo, can lead to a perturbation 


of (y) = acos ny 


in f(y) which remains of finite magnitude a for all n. This means that the 
solution f(y) does not depend continuously on the data g(a), and so the 
problem is ill-posed. 


We thus see that it is not in fact necessarily advantageous for the func- 
tion K to be smooth. Nevertheless, there are ways of obtaining acceptable 
numerical solutions to problems such as (9.20). They are based on the tech- 
nique of regularisation, which effectively forces an approximate solution to be 
appropriately smooth. We return to this topic in Section 9.6 below. 
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9.5 Singular kernels 


A particularly important class of kernels, especially in the context of the study 
of Chebyshev polynomials in integral equations, comprises the Hilbert kernel 


1 
ty 


K(a2,y) = (9.21) 


and other related ‘Hilbert-type’ kernels that behave locally like (9.21) in the 
neighbourhood of x = y. 


9.5.1 Hilbert-type kernels and related kernels 


If [a, b] = [—1, 1] and 
w(y) 


oe 


K (x,y) = 


where w(y) is one of the weight functions (1 + y)*(1— y)® with a,8 = +3, 
then there are direct links of the form (9.20) between Chebyshev polynomials 
of the four kinds (Fromme & Golberg 1981, Mason 1993). 


Theorem 9.1 
TUn—1(£) = f Ki(a,y)Tr(y) dy, (9.22a) 
—1T, (xz) = - Ko(x,y)Un_1(y) dy, (9.22b) 
TW, (x) = f K3(x,y)Vn(y) dy, (9.22c) 
—1V, (2) = f K4(x,y)Wrly) dy (9.22d) 
where 
ee ons 
K2(x,y) = ok 
Ka(ey) = 
Ka(y) = =e, 


and each integral is to be interpreted as a Cauchy principal value integral. 
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Proof: In fact, formulae (9.22a) and (9.22b) correspond under the transformation 
x = cos@ to the trigonometric formulae 


*  cosnd sin n0 
SOR Sas 2 
f cos ¢ — cos 0 ? 7 sind” 2) 
*™ sinndsin bd 
—— dg = - é 9.24 
f cos d — cos 0 % pee (9:24) 


which have already been proved in another chapter (Lemma 8.3). Formulae (9.22c) 
and (9.22d) follow similarly from Lemma 8.5. @@ 


From Theorem 9.1 we may immediately deduce integral relationships be- 
tween Chebyshev series expansions of functions as follows. 


Corollary 9.1A 
1. If f(y)~ a AnTn(y) and g(x) ~ Ta AnUn—1(x) then 


dy. (9.25a) 


ae I, Play yas) 
2. If f(y) ~ er bnUn-1(y) and g(x) ~ ae brTp(x) then 
--f ow a dy. (9.25b) 


3. If fly) ~ 24 CnValy) and g(x) ~ tL; énWa(z) then 
oT) 3 


MO | =90- 5) Ve) 
4. I$) ~ 021 dnWo(y) and gl) ~ 72 daVa (2) then 
© VIF fy) d (9.25d) 


i am 1 VI Fy (y-2) 


Note that these expressions do not necessarily provide general solutions to 
the integral equations (9.25a)—(9.25d), but they simply show that the relevant 
formal expansions are integral transforms of each other. 


These relationships are useful in attacking certain engineering problems. 
Gladwell & England (1977) use (9.25a) and (9.25b) in elasticity analysis and 
Fromme & Golberg (1979) use (9.25c), (9.25d) and related properties of V, 
and W,, in analysis of the flow of air near the tip of an airfoil. 


To proceed to other kernels, we note that by integrating equations (9.22a)— 
(9.22d) with respect to x, after premultiplying by the appropriate weights, we 
can deduce the following eigenfunction properties of Chebyshev polynomials 
for logarithmic kernels. The details are left to the reader (Problem 4). 
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Theorem 9.2 The integral equation 


do(e) = / Ke dy (9.26) 


has the following eigensolutions and eigenvalues » for the following kernels 


y) = log|y — a]; 
$(x) = bn(x) =Th(x), A=An=T/n. 


2. K(x,y) = Ke(x,y) = log ly — 2] — log 1 — ay — (1 — 27)(1 — y?)); 
O(a) = bn(a) = V1 — a? Un_-i1(2), A=Ay=T/N. 


3. K(a,y) = K7(a,y) = log|y —2| —log|2+a+y—-2Vl4+aV1+y); 


b= on(z) = VIF EV, (2), A=An =7/(n+4 4). 
4. K(x, y) = Ke(a,y) = log|y — z| — log ]2-—-a—y-—2V1—aV/1—y]; 


Note that each of these four kernels has a (weak) logarithmic singularity 
at x = y. In addition, Ke has logarithmic singularities at 7 = y= +1, K7 at 
x=y=-—land Kg atxv=y=+1. 


From Theorem 9.2 we may immediately deduce relationships between for- 
mal Chebyshev series of the four kinds as follows. 


Corollary 9.2A With the notations of Theorem 9.2, in each of the four cases 
considered, if 


f(y) ~ do ande(y) and g(x) ~ S> Acande(z) 
k=1 k=1 


then 
g(a) = i ee msde. 


Thus again four kinds of Chebyshev series may in principle be used to solve 
(9.26) for K = Ks, Ke, K7, Kg, respectively. 


The most useful results in Theorem 9.2 and its corollary are those relating 
to polynomials of the first kind, where we find from Theorem 9.2 that 


T I 1 
——T), (x =| ——-T,, (y) lo —a\d 9.27 
7 ile) Le (y) log |y — 2| dy (9.27) 
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and from Corollary 9.2A that, if 


y)~ Yo acLi(y) and g(x) ~ > -FaxTi(a), (9.28) 
= k=1 
then 
1 1 
y= f Seals —alsew (9.29) 


Equation (9.29) is usually referred to as Symm’s integral equation, and 
clearly a Chebyshev series method is potentially very useful for such problems. 
We shall discuss this specific problem further in Section 9.5.2. 


By differentiating rather than integrating in (9.22a), (9.22b), (9.22c) and 


(9.22d), we may obtain the further results quoted in Problem 5. The second 
of these yields the simple equation 


ja) ry, 


—ntUy_1( _i(y) dy. (9.30) 
This integral equation, which has a stronger singularity than (9.22a)—(9.22d), 
is commonly referred to as a hypersingular equation, in which the integral 
has to be evaluated as a Hadamard finite-part integral (Martin 1991, for 
example). A rather more general hypersingular integral equation is solved by 
a Chebyshev method, based on (9.30), in Section 9.7.1 below. 

The ability of a Chebyshev series of the first or second kind to handle 
both Cauchy principal value and hypersingular integral transforms leads us 
to consider an integral equation that involves both. This can be success- 
fully attacked, and Mason & Venturino (2002) give full details of a Galerkin 
method, together with both £2 and £L error bounds, and convergence proofs. 


9.5.2 Symm/’s integral equation 


Consider the integral equation (Symm 1966) 


b 
G(a) = VF (x) = ~ | log |y—a|F(y)dy, «x € [a,b], (9.31) 


which is of importance in potential theory. 


This equation has a unique solution Fy) (Jorgens 1970) with endpoint 
singularities of the form (y —a)~2(b — y)~?. In the case a = —1, b = +1, the 
required singularity is (1 — y?)~2, and so we may write 


Fly) =(1—y*)"?f(y), G(x) = —171 (2), 
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whereupon (9.31) becomes 


aya vse) =f SE roan 


which is exactly the form (9.29) obtained from Corollary 9.2A. 
We noted then (9.29) that if 


y) ~ do axTe(y) 
k=1 
then 


oO 
~ y ——agT, (x 
k 
k=1 
(and vice versa). 
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(9.32) 


Sloan & Stephan (1992), adopt such an idea and furthermore note that 


V*To(x) = —Tlog 2, 


so that ws 
/ 
y) ~ d5 axTe(y) 
k=0 
if Zz 
7 
g(x) ~ —$aom log 2 — > 7 oe li(@): 


Their method of approximate solution is to write 


n—1 
y) = D> afta) 
k=0 
and to require that 
v*f*(x) = g(a) 
holds at the zeros x = x; of T,(a«). Then 


n-1 


* Ts - 
g(xi) = — gag log2 — S| pax Te (2), i=0,...,n—1. 


k=1 


Using the discrete orthogonality formulae (4.42), we deduce that 


a, = —— g(ai)Te(ai) (k > 0). 


(9.33) 


(9.34) 


(9.35) 


(9.36) 
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Thus values of the coefficients {a} are determined explicitly. 


The convergence properties of the approximation f* to f have been estab- 
lished by Sloan & Stephan (1992). 


9.6 Regularisation of integral equations 


Consider again an integral equation of the first kind, of the form 
=f K(a,y)f(y) dy, c<a<d, (9.37) 


ie. g = Kf, where K : F — G and where the given g(x) may be affected by 
noise (Bennell & Mason 1989). Such a problem is said to be well posed if: 


e for each g € G there exists a solution f € FP; 
e this solution f is always unique in F; 


e f depends continuously on g (i.e., the inverse of K is continuous). 


Unfortunately it is relatively common for an equation of the form (9.37) to 
be ill posed, so that a method of solution is needed which ensures not only that 
a computed f is close to being a solution but also that f is an appropriately 
smooth function. The standard approach is called a regularisation method; 
Tikhonov (1963), 1963a) proposed an Lz approximation which minimises 


b 
Ii f*] =) [Kf*(z) — Parr [we )? + q(x) f*’(ax)?] dx (9.38) 


where p and q are specified positive weight functions and \ a positive ‘smooth- 
ing’ parameter. The value of A controls the trade-off between the smoothness 
of f* and the fidelity to the data g. 


9.6.1 Discrete data with second derivative regularisation 


We shall first make two changes to (9.38) on the practical assumptions that we 
seek a visually smooth (i.e., twice continuously differentiable) solution, and 
that the data are discrete. We therefore assume that g(x) is known only at n 
ordinates «; and then only subject to white noise contamination €(x;); 


*(x;) -[ K (ai, y)f (y) dy + €(a;) (9.39) 


where each ¢(x;) ~ N(0,07) is drawn from a normal distribution with zero 
mean and (unknown) variance 0”. We then approximate f by the f* € Le[a, >| 
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that minimises 
1 nm 
J=- y [Kf* (ai) — g*(a;)] afl Fu * dy, (9.40) 
n 
i=1 


thus replacing the first integral in (9.38) by a discrete sum and the second by 
one involving the second derivative of f*. 


Ideally, a value Aop_ of A should be chosen (in an outer cycle of iteration) 
to minimise the true mean-square error 


RQ) = = DKA (2) — gaa) (9.41) 


This is not directly possible, since the values g(x;) are unknown. However, 
Wahba (1977) has shown that a good approximation to A>p_ may be obtained 
by choosing the ‘generalised cross-validation’ (GCV) estimate \3,, that min- 


imises ; 
+ ||I— AQ)gl 


vay = at Ae 
[2 trace(I — A(A))] 


(9.42) 


where 
Kfy = A()g, (9.43) 


i.e., A(A) is the matrix which takes the vector of values g(a;) into K fx (a;). 


An approximate representation is required for f¥. Bennell & Mason (1989) 
adopt a basis of polynomials orthogonal on [a,b], and more specifically the 
Chebyshev polynomial sum 


(y) = )5 aT; (y) (9.44) 
j=0 
when [a,b] = [—1, 1]. 


9.6.2 Details of a smoothing algorithm (second derivative regular- 
isation) 


Adopting the representation (9.44), the smoothing term in (9.40) is 


/ LW]? dy =a? Ba (9.45) 


where 4 = (a2, 03,...,@m)! and B is a matrix with elements 


1 
Bye / Pl (y)P"(y)dy (6,7 =2,...)m). (9.46) 
—1 
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The matrix B is symmetric and positive definite, with a Cholesky decompo- 
sition B = LL’, giving 


2 


[oP ey = 0% 


Then, from (9.40), 


irks ‘ 4/2 
I fx] = s |[Ma — g*|? + \||L7a 


(9.47) 


where M;; = li K (aj, y)T;(y) dy and a = (a1, 42,...,@m)?. 


Bennell & Mason (1989) show that ao and a; may be eliminated by con- 
sidering the QU decomposition of M, 


M=QU=Q | (9.48) 


where Q is orthogonal, V is upper triangular of order m+ 1 and then 


U= | a | (9.49) 


where R, is a 2 x 2 matrix. 


Defining 4 = (ao, a1)’, 


Ma — g*||? = ||QUa — g* ||’ 
2 


= ||Q7(QUa — g*) 
= [Ua — el]? , where e = Q’g*, 
= ||R1a + Roa — ||’ + ||R34— él. (9.50) 


Setting a = R;'(e — Raa), 


2 


1 A “a A 
[fx] = = |[Roa — 6? +A||L"A (9.51) 


The problem of minimising J over 4 now involves only the independent 
variables a2,...,@m, and requires us to solve the equation 
(H7H + nXAI)b = H7é (9.52) 


where b = L’A and H = R;(L7)-!. 


Hence 
b = (H?H + nXI)'H76 (9.53) 


89.6: REGULARISATION OF INTEGRAL EQUATIONS 217 


and it can readily be seen that the GCV matrix is 
A(\) = H(H7H + nXI)~'H". (9.54) 
The algorithm thus consists of solving the linear system (9.52) for a given 
while minimising V(A) given by (9.42). 
Formula (9.42) may be greatly simplified by first determining the singular 
value decomposition (SVD) of H 
H = WAX" (W, X orthogonal) 


gihiete 
ie i (A = diag(d;)). 


ve, Pe AC aid) 1222 Be 
V(A) = SebsL i FATT 2 F ae Fk (9.55) 
hey Ad? + ndA)~! + (n —m-—1)n7! 
where z = W7€. 

The method has been successfully tested by Bennell & Mason (1989) on a 
number of problems of the form (9.37), using Chebyshev polynomials. It was 
noted that there was an optimal choice of the number m of basis functions, be- 
yond which the approximation fy deteriorated on account of ill-conditioning. 
In Figures 9.1-9.3, we compare the true solution (dashed curve) with the 
computed Chebyshev polynomial solution (9.44) (continuous curve) for the 
function f(y) =e ¥ and equation 


CoO 1 
—y dy=——, 0<2<0, 
i e 4 f(y) dy es xr < OO 


with 
e (x) ~ N(0,.0057) and m= 5, 
e <(z) ~ N(0,.017) and m=5, 
e <(x) ~ N(0,.017) and m = 10. 
No significant improvement was obtained for any other value of m. 
9.6.3 A smoothing algorithm with weighted function regularisa- 
tion 


Some simplifications occur in the above algebra if, as proposed by Mason & 
Venturino (1997), in place of (9.40) we minimise the functional 


b 
IFISS Ke) - ae) +f wl way. (9.56) 
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Figure 9.1: Data error N(0, 0.0057); 5 approximation coefficients 


Figure 9.2: Data error N(0, 0.017); 5 approximation coefficients 


Figure 9.3: Data error N(0, 0.017); 10 approximation coefficients 
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This is closer to the Tikhonov form (9.38) than is (9.40), and involves weaker 
assumptions about the smoothness of /. 


Again we adopt an orthogonal polynomial sum to represent f*. We choose 
w(a) to be the weight function corresponding to the orthogonality. In par- 
ticular, for the first-kind Chebyshev polynomial basis on [—1, 1], and the ap- 
proximation 


KY) = >_ a5T5(y), (9.57) 
j=0 


the weight function is of course w(x) = 1/V1 — «?. 


The main changes to the procedure of Section 9.6.2 are that 


e in the case of (9.56) we do not now need to separate ag and a, off from 
the other coefficients a,, and 


e the smoothing matrix corresponding to B in (9.46) becomes diagonal, 
so that no LL? decomposition is required. 


For simplicity, we take the orthonormal basis on [—1, 1], replacing (9.57) with 


FRY) = d¢ 45;(y) = S55 [T)(y)/n3], (9.58) 
j=0 j=0 
where 
n? = { is coe (9.59) 


Define two inner products (respectively discrete and continuous); 


n 


(u, Ug = S/ u(xe)v(xr); (9.60) 
k=1 
(u,v). = [ we@ua@oa da. (9.61) 


Then the minimisation of (9.56), for f* given by (9.58), leads to the system 
of equations 


S > a; (Kbi, Kb) g — (KG: Bg t+ rrAY a; (bi, $j), =0, 1=0,...,m. 

j=0 j=0 

; , (9.62) 
Hence 


(Q7Q + nAl)a = Q’ g* (9.63) 
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(as a consequence of the orthonormality), where 


Qk,j = (Koj) (wr) (9.64) 


and a, g* are vectors with components a; and g*(x;), respectively. 


To determine f* we need to solve (9.63) for aj, with A minimising V(A) 
as defined in (9.42). The matrix A()) in (9.42) is to be such that 


Kt = A(A)g*. (9.65) 
Now Kf* = 1 5 ajKoy (zx) = Qa and hence, from (9.63) 


A(A) = Q(Q7Q + nAT)'Q?. (9.66) 


9.6.4 Evaluation of V()) 


It remains to clarify the remaining details of the algorithm of Section 9.6.3, 
and in particular to give an explicit formula for V(A) based on (9.66). 
Let 
Q = WAX* (9.67) 


be the singular value decomposition of Q, where W is n x n orthogonal, X is 
(m+ 1) x (m+ 1) orthogonal and 


ne Am | (9.68) 
x (m+1) 
Define 
z= [zx] = W’g. (9.69) 
From (9.68), 
ATA = diag(d3,..., d?,). (9.70) 
It follows that 
A(\) = WB(A)W* (9.71) 
where 
B(A) = A(ATA 4 ndAl)~1 AT (9.72) 
so that B()) is the n x n diagonal matrix with elements 
d2 


= k 
d?+nr 
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From (9.71) and (9.72) 
| — AQ) gl? = | — AQ) Wall? 
= ||W7(I— A(\)) Wall” 
= ||@- WT AQ)w)z ||” 


= || -B()))all’ 
m nd 2 . n—-1 ‘ 
=D (aa) a+ Da 
i=0 i=m+1 
Thus 
2 — AQ) gl? = Lonel eae (9.74) 
i=m4+1 
where 
e;(A) d+ nA (9.75) 
Also 
trace(I — A(A)) = trace W7 (I— A(\))W 
= trace(I — B(A)) 
= 1. 
- Lam aaa > 
Thus 
trace(I— A = Yonted (n-—m-—1). (9.76) 


Finally, from (9.75) and (9.76), together with (9.42), it follows that 


(9.77) 


9.6.5 Other basis functions 


It should be pointed out that Chebyshev polynomials are certainly not the 
only basis functions that could be used in the solution of (9.37) by regulari- 
sation. Indeed there is a discussion by Bennell & Mason (1989, Section ii) of 
three alternative basis functions, each of which yields an efficient algorithmic 
procedure, namely: 
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1. a kernel function basis {K(x;, y)}, 
2. a B-spline basis, and 


3. an eigenfunction basis. 


Of these, an eigenfunction basis is the most convenient (provided that eigen- 
functions are known), whereas a kernel function basis is rarely of practical 
value. A B-spline basis is of general applicability and possibly comparable 
to, or slightly more versatile than, the Chebyshev polynomial basis. See Ro- 
driguez & Seatzu (1990) and also Bennell & Mason (1989) for discussion of 
B-spline algorithms. 


9.7 Partial differential equations and boundary integral equation 
methods 


Certain classes of partial differential equations, with suitable boundary con- 
ditions, can be transformed into integral equations on the boundary of the 
domain. This is particularly true for equations related to the Laplace oper- 
ator. Methods based on the solution of such integral equations are referred 
to as boundary integral equation (BIE) methods (Jaswon & Symm 1977, for 
instance) or, when they are based on discrete element approximations, as 
boundary element methods (BEM) (Brebbia et al. 1984). Chebyshev polyno- 
mials have a part to play in the solution of BIEs, since they lead typically to 
kernels related to the Hilbert kernel discussed in Section 9.5.1. 


We now illustrate the role of Chebyshev polynomials in BIE methods for a 
particular mixed boundary value problem for Laplace’s equation, which leads 
to a hypersingular boundary integral equation. 


9.7.1 A hypersingular integral equation derived from a mixed 
boundary value problem for Laplace’s equation 
Derivation 


In this section we tackle a ‘hard’ problem, which relates closely to the hyper- 
singular integral relationship (9.30) satisfied by Chebyshev polynomials of the 
second kind. The problem and method are taken from Mason & Venturino 
(1997). 


Consider Laplace’s equation for u(x, y) in the positive quadrant 
Au=0, 2,y>0, (9.78) 


subject to (see Figure 9.4) 
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u=0 / wu bounded 


Figure 9.4: Location of the various boundary conditions (9.79) 


u(x,0)=0, «>0, (9.79a) 

hu(0,y) + ue(0,y) = gly), O< asySb, (9.79b) 
u(0,y) =0, 0< y<a; b<y, (9.79c) 

u(x, y) is bounded, 2, y > ow. (9.79d) 


Thus the boundary conditions are homogeneous apart from a window L = 
(a, b] of radiation boundary conditions, and the steady-state temperature dis- 
tribution in the positive quadrant is sought. Here the boundary conditions 
are ‘mixed’ in two senses: involving both u and u, on L and splitting into two 
different operators on xz = 0. Such problems are known to lead to Cauchy sin- 
gular integral equations (Venturino 1986), but in this case a different approach 
leads to a hypersingular integral equation closely related to (9.30). 


By separation of variables in (9.78), using (9.79a) and (9.79d), we find 
that 


u(e,y) =f A(u)sin(q1y)exp(—na) dy (9.80) 
0 
The zero conditions (9.79c) on the complement L*° of L give 


u(0,y)= lim f A(u)sin(uy) exp(—va)du=0, ye L’, (9.81) 
it bia (0) 
and differentiation of (9.80) with respect to x in L gives 


ue(0,y) = — lim, ' LA(p) sin(wy) exp(—px)du=0, yeL. (9.82) 
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Substitution of (9.80) and (9.82) into (9.79b) leads to 


CO 


jim, ; (h — p)A() sin(wy) exp(—px) du = gly), ye L. (9.83) 


Then (9.81) and (9.83) are a pair of dual integral equations for A(j), and 
from which we can deduce wu by using (9.80). 


To solve (9.81) and (9.83), we define a function B(y) as 
Biv) = ul0,y) =f Aw)sin(uy) adn, y 20 (9.84) 
0 
Then, from (9.81) 


By) =0, yelL, (9.85) 


and, inverting the sine transform (9.84) and using (9.85), 
: B(t) sin(st) dt = $7A(s). (9.86) 
L 


Substituting (9.86) in the integral equation (9.83) gives us 


nBty) = | 114) at =a) ved (9.87) 
where 
I(t) = lim Pe sin ut exp(—pux) dys 
= 3 lim, ce [cos u(t — y) — cos u(t + y)] exp(—px) du. (9.88) 


This simplifies (see Problem 7) to 


S$ yee | Sea 2 ey) 
I(t) = 5 lim, pt (2? + (t+y)?)? 
ee ah (9.89) 
(t—y)2 (t+y) 


Substituting (9.89) into (9.87), we obtain the hypersingular integral equa- 
tion, with strong singularity at t = y, 
nsw) += | Bo| : : Ja (y), yeL, — (9.90) 
y = Tap BAD PE ENS: =Gy), Y¥ , . 
TIL (t—y)? (t+y)? 


from which B(y) is to be determined, and hence A(s) from (9.86) and u(z, y) 
from (9.80). 
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Method of solution 


Continuing to follow Mason & Venturino (1997), equation (9.90) can be rewrit- 
ten in operator form as 


Ag =(h+H+K)¢= f (9.91) 


where H is a Hadamard finite-part integral and K is a compact perturbation, 
given by 


(H¢)(x) = i o(s) dey “eee, (9.92) 


a) 


(Ko)(x) = [| K(w,8)0(s) ds =f wh, ds, -l<2<1l, (9.93) 


and 
f(z) =9 ($(b-a)x + $(b+a)). (9.94) 


It is clear that ¢(2) must vanish at the end points +1, since it represents 
boundary values, and moreover it should possess a square-root singularity 
(Martin 1991). Hence we write 


g(x) = w(x)y(x), where w(x) = V1 - 2?. (9.95) 


We note also that the Hadamard finite-part operator maps second kind 
Chebyshev polynomials into themselves, as shown by Mason (1993) and Mar- 
tin (1992) and indicated in (9.30) above; in fact 


H(wUe)(x) = —m(€+1)Ud(2), &>0. (9.96) 


Solution of (9.86) in terms of second-kind polynomials is clearly suggested, 


namely 
CO 


y(x) = S$ eUe(z), (9.97) 


£=0 
where the coefficients cg are to be determined, and we therefore define a 
weighted inner product 


and observe that 
Well, = 37, 220. (9.98) 


We also expand both f(x), the right-hand side of (9.91), and K(a,t) in 
second-kind polynomials 


f(x) = © fiU;(z), where f; = - (FU (9.99) 
1=0 
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= Sl Kytilauzeo, (9.100) 


i=0 j=0 
so that (9.93), (9.97) and (9.98) give 


= DODD eK ig Uae) [Uslly, d5e 


1=0 7=0 €=0 
= $n) oc S) KuUi(a). (9.101) 
Substituting (9.95), (9.97), (9.99), (9.101) and (9.99) into (9.91): 


hw J exte(a) yams (2+ VceUe(x y+ Deed Kuti = Dus 
L=0 = 


£=0 
(9.102) 
Taking the weighted inner product with U;: 


hy- ce (wu, , U5). = Soe + 1)ce (Ue, U5) + 
= £=0 


lo) co 1 
+ sm ce) Kie (U; ; Uz), = gt hi- (9.103) 
L=0 i 


Define F 
peau. =| (1 — 22)Ue(2)U;(2) de. (9.104) 
-1 
Then it can be shown (Problem 8) that 
d + £ eve 
TF Oo even 
be=< (U+j+2°-1 (@-j2-1’ ” (9.105) 
0, otherwise. 
Hence, from (9.103), 
Die gm (J+ Le; + ($n)? So Kjece = ati 0<j<co. (9.106) 
L=0 


Reducing (9.106) to a finite system, to solve for approximate coefficients ¢,, 
we obtain 
N-1 N-1 
hS© djeee—3 0? G+ Le +($0)? 2 K jee = ati 0 <j <N-1. (9.107) 
£=0 l= 
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Table 9.1: Results for K = 0, d(x) = V1 — 2? expx 
Tew 
1 1.93 
2.81 


4.17 
6.54 
10.69 


EXAMPLE 9.1: The method is tested by Mason & Venturino (1997) for a slightly 
different problem, where the well-behaved part K of the problem is set to zero and 
the function f is chosen so that ¢(x) = V1 — #7 exp. The condition number of the 
matrix of the linear system (9.107) defining é; is compared in Table 9.1 with the 
maximum error |len||,, for various values of N, and it is clear that the conditioning 
is relatively good and the accuracy achieved is excellent. 


Error analysis 


A rigorous error analysis has been carried out by Mason & Venturino (1997), 
but the detail is much too extensive to quote here. However, the conclusion 
reached was that, if f € C?*![-1,1] and the integral operator K satisfies 
certain inequalities, then the method is convergent and 


len log < C.N7~ PD (9.108) 


where the constant C depends on the smoothness of K and f but not on N. 


For further studies of singular integral equations involving a Cauchy kernel, 
see Elliott (1989) and Venturino (1992, 1993). 


9.8 Problems for Chapter 9 
1. Follow through all steps in detail of the proofs of Theorem 9.1 and 
Corollary 9.1A. 
2. Using Corollary 9.1A, find a function g(x) such that 


Lf VIO) 


ue) = -1 (y—2) 


y 


in the cases 
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(a) f(y) =1; 
(b) f(y) =y°: 
(c) fly) =e% 
(d) fy) = V1—y9?. 
3. Using Corollary 9.1A, find a function g(a) such that 


a a ely 
a V1-yy- 2) 
in the cases 


(a) g(a) =e"; 
(b) g(w) =(1+2)?(1— 2)~3; 


. Prove Theorem 9.2 in detail. For instance, the second kernel Ke in the 


theorem is derived from 
“ Ko(x,y) 
Se 1- x2 


Setting x = cos2¢, y = cos 2 and tan ¢@ = t, show that Ke simplifies to 


_ sin(é + wv) 
om a fF sin? Se cos? wy =A0e sin(@ — yw) | 


Then, by setting x = 2u? —1, y = 2v? — 1 and noting that V1 — 2? = 
2uV1 —u?, 1 — y? = 2uvV/1 — v2, show that K6(x, y) simplifies to 


log |x — y| log |1 wy ayo? NCL. P|. 


Ke(x,y) = de. 


. By differentiating rather than integrating in (9.22a), (9.22b), (9.22c) 


and (9.22d), and using the properties 


[V1 —2?U,-1(x)]' = —nT,(x)/V1 — 2?, 
[Tn(x)]' = nUn-r(2), 
[V1 — a2 W(x)’ = (n+ 3)Vn(2)/V1— 2, 
| 


[V1 +2Vn(x))’ = (n+ 4)Wr(2)/V1 +2, 


deduce that the integral equation 


K (a, y) dy 


1 
= f =e 
-14/l-—- y? 
has the following eigensolutions ¢ and eigenvalues 4 for the following 
kernels K: 
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eee x 
(a) K(2z,y) = Ko(z,y) = (==. G==GeD. 
@ = on(£) = Th-1(4)/V1— #7, X= An = —1T 
(0) K(@,9) = Kw(eu) = 74 
ob = o,(x) = Un-1(2), A= An = —NT. 
2 —_ VYd-a)1+y) IT+y 
(c) K(az,y) = Kii(z,y) = (y a)? = 2 T—x(y —2) 
o= gn(2) = Vn (x) /v 1-2, X}\=An = =e 5)m 
= VTi ae) lag .. 
(d) FY) Ge ans rama 


b= bn(x) = W,, (2) /J/1 +2, A= An = —(n + 4)n. 


6. (a) Describe and discuss possible amendments that you might make to 
the regularisation methods of Section 9.6 in case kK has any one 
of the four singular forms listed in Theorem 9.1. Does the method 
simplify? 

(b) Discuss whether or not it might be better, for a general K, to use 
one of the Chebyshev polynomials other than T;(«) in the approx- 
imation (9.44). 


7. Show that 


- Potayy. way 
sin pt si exp(—px) du = —— >. - — 5: 
fina sin exol—18) n= CG a” GEE 
(This completes the determination of I(t), given by (9.88), so as to 
derive the hypersingular equation (9.90).) 


8. Show that 
fo 2)Up(2)U; (a) dr = : gated 
hype One ace Gage Qe. eng ea 


for €+ 9 even, and that the integral vanishes otherwise. (This is a step 
required in the derivation of the solution of the hypersingular equation 
(9.90).) 


CHAPTER 10 


Solution of Ordinary Differential Equations 


10.1 Introduction 


While, historically, finite-difference methods have been and remain the stan- 
dard numerical technique for solving ordinary differential equations, newer al- 
ternative methods can be more effective in certain contexts. In particular we 
consider here methods founded on orthogonal expansions—the so-called spec- 
tral and pseudospectral methods—with special reference to methods based on 
expansions in Chebyshev polynomials. 


In a typical finite-difference method, the unknown function u(x) is repre- 
sented by a table of numbers {yo, y1,.--,; Yn} approximating its values at a set 
of discrete points {vo,71,...,%n}, so that y; % u(x;). (The points are almost 
always equally spaced through the range of integration, so that 7;41 —2; =h 
for some small fixed h.) 


In a spectral method, in contrast, the function u(«) is represented by an 
infinite expansion u(x) = >>, ce@r(x), where {¢,} is a chosen sequence of 
prescribed basis functions. One then proceeds somehow to estimate as many 
as possible of the coefficients {c,}, thus approximating u(x) by a finite sum 
such as 


Un(x) = S > cen (a). (10.1) 
k=0 


One clear advantage that spectral methods have over finite-difference meth- 
ods is that, once approximate spectral coefficients have been found, the ap- 
proximate solution can immediately be evaluated at any point in the range of 
integration, whereas to evaluate a finite-difference solution at an intermediate 
point requires a further step of interpolation. 


A pseudospectral method, at least according to some writers, is one in 
which u(x) is still approximated by a function of the form u,(x) of (10.1), as 
in a spectral method, but this approximation is actually represented not by 
its coefficients but by its values u,,(x,;) at a number (n+ 1 in this particular 
instance) of discrete points {x;}. These points may be equally spaced, but 
equal spacing gives no advantages, and other spacings are frequently better. 


The oldest and probably the most familiar spectral methods are based on 
the idea of Fourier series. Supposing for the moment, for convenience, that the 
independent variable x is confined to the interval —7 < x < 7, the technique 
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is to assume that the unknown function has an expansion in the form 


u(a) = $a9 + S— {ax cos kar + by sin kar} (10.2) 
k=1 


and to attempt to determine values for the coefficients {ax, b,} such that the 
required differential equation and other conditions are satisfied. 


Fourier methods may well be suitable when the problem is inherently peri- 
odic; for instance where the function u(x) satisfies a second-order differential 
equation subject to the periodicity boundary conditions u(a) = u(—7) and 
u'(a) = u'(—7). If we have a second-order differential equation with the more 
usual boundary conditions u(—a) = a and u(m) = b, however, with a F 5b, 
then obviously any finite partial sum 


n 
$a) + Side cos ka + by sin ka} 
k=1 


of (10.2) is periodic and cannot satisfy both boundary conditions simultane- 
ously; more importantly, very many terms are needed if this partial sum is 
to represent the function u(a) at all closely near both ends of the range at 
the same time. It is better in such a case to take for {¢,} a sequence of 
polynomials, so that the partial sum 


se cebr(2) 
k=0 


is a polynomial of degree n. 


From now on, we shall suppose that the independent variable x is confined 
to the interval —-1 < x < 1, so that a reasonable choice for ¢,(x) is the 
Chebyshev polynomial T;,(a). The choice of basis is only the beginning of the 
story, however; we are still left with the task of determining the coefficients 


{cx}. 
10.2 A simple example 
For the purposes of illustration, we shall consider the simple linear two-point 


boundary-value problem on the range [—1, 1]: 


d2 
u(t) =f(@), w(-=a, u(+1) =, (10.3) 
where the function f and the boundary values a and b are given. 


We can start out in several ways, of which the two following are the sim- 
plest: 
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e We may write u(2) directly in the form 


oe / 
u(x) = SS Cel (2). (10.4) 
k=0 
Using the result quoted in Problem 16 of Chapter 2, namely 
2 k-2 
qua Tele) = So (k-r)k(k+r)T,(a), (k > 2), (10.5) 


r=0 
(k—r) even 


we express (10.3) in the form 


6o,  k=2 
So SS (h-r)k(k + r)ceT, (a) = f(a), (10.6a) 
NS feo es 
S~(-1)Fer =a, So ce =o. (10.6b) 
k=0 k=0 


e An alternative procedure, incorporating the boundary conditions in the 
representation itself, is to write u(x) in the form 


= ie 1 
ule) = (1-27) Ue (x) + Sato. 00.7) 
k=0 


Then, using a result given from Problem 12 of Chapter 3, 
(1 — 27)Ug (2) = 5(Tk(2) — Tr+2(z)), 
together with (10.5), we get (10.3) in a single equation of the form 


ae (k—r)k(k +1) (Vk — Ye-2)Tr(x) = f(a). (10.8) 


We should be treading on dangerous ground, however, if we went ahead 
blindly with translating either of the above infinite systems of equations (10.6) 
or (10.8) into an algorithm, since non-trivial questions arise relating to con- 
vergence of the infinite series and the validity of differentiating term by term. 
In any event, since one can never calculate the whole of an infinite sequence of 
coefficients, the realistic approach is to accept the fact that we must perforce 
approximate u(x) by a finite sum of terms, and to go on from there. 


If we truncate the summation (10.4) or (10.7) after a finite number of 
terms, we obviously cannot in general satisfy (10.6a) or (10.8) throughout 
the range —1 < x < 1. We can, however, attempt to satisfy either equa- 
tion approximately in some sense. We shall discuss two ways of doing this: 
collocation methods and projection or tau (7) methods. 
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10.2.1 Collocation methods 


Suppose that we approximate u(x) by 


un(a) = S> eT (a), (10.9) 
k=0 


involving n + 1 unknown coefficients {c;,}, then we may select n — 1 points 
{v1,...,%,-1} in the range of integration and require un(x) to satisfy the 
differential equation (10.3) at just these n —1 points, the so-called collocation 
points, in addition to obeying the boundary conditions. This requires us to 
solve just the system of n + 1 linear equations 


k-2 


» (k—r)k(k + r)ceT (aj) = f(%;); 


S~ (-1)kex = a, (10.10b) 


k=0 


(10.10c) 


i) 
cad 

II 
— 


These equations may be reduced to a simpler form, especially if the n — 
1 points are carefully chosen so that we can exploit discrete orthogonality 
properties of the Chebyshev polynomials. Suppose that we choose for our 
collocation points the zeros of T,_1(z), namely 


Xj = COs (10.11) 
Multiply (10.10a) by 27;(z,;), where @ is an integer with 0 < @< n— 2, and 
sum from 7 = 1 to 7 = n—1. We can then use the discrete orthogonality 
relations (4.42) (forO<r<n-—2,0<¢<n-2) 


n-1 n—-1, r=l= 0, 
S°T.(x;)Te(a;)=4 $(n—-1), r=l¥40, (10.12) 
j=l 0, otherwise 


to deduce that 


n 


n-1 
2 
So (kD) (K+) ce = a S > Te(a,)f(a;), £=0,...,m—2. (10.13) 
a Ag j=l 
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The matrix of coefficients on the left-hand side of equation (10.13) is upper 
triangular, with elements 


8 64 216 512 1000 

24 120 336 720 

48 192 480 960 
80 280 648 

120 384 840 
168 504 

224 640 
288 

360 


We now have the following algorithm for generating an approximate solu- 
tion of the problem (10.3) by collocation. 


2 
1. Find the collocation points x; = cos G2) z 


1, and evaluate f(x;); 


,forj =1,...,n- 


2. Use the recurrence (1.3) to evaluate Ty(a;), for = 0,...,n—2 
and 7 =1,...,n—1; 


3. Use equations (10.13), in reverse order, to determine the co- 


efficients Cn,..., C3, C2 one by one; 
4. Use the boundary conditions (10.10b), (10.10c) to determine 
Co and cj. 


This algorithm can be made considerably more efficient in the case where 
n — 1 is a large power of 2, as we can then use a technique derived from the 
fast Fourier transform algorithm to compute the right-hand sides of (10.13) 
in O(nlogn) operations, without going through step 2 above which requires 
O(n) operations (see Section 4.7). 


Alternatively, we approximate u(x) by 


1-2 gle 
a 
2 De ve 


Un(x) = (1-27) > eo) - (10.14) 
k=0 


involving n — 1 unknown coefficients {y,} and satisfying the boundary con- 
ditions automatically. With the same n— 1 collocation points {x1,...,%n—1} 
we now solve the system of n — 1 linear equations 


n k-2 
SS Ss a(k—r)k(k +1) (Ye — Ye—2)Tr (23) = f (a5) (10.15) 
k=2 r=0 


(k—r) even 
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(with Yn = Yn—1 = 0). If the collocation points are again taken as the zeros 
of T,-1(x), discrete orthogonality gives the equations 


n 


n-1 
S> FR - ORE T+ Oa) = Tle) flas) (10.16) 
Gas = 


or, equivalently, 
n—2 2 n-1 
2 2 _ ; ; = =: 


k= 
(k—£) even 
(10.17) 


The matrix of coefficients on the left-hand side of equation (10.17) is upper 
triangular, with elements 


—4 —28 —76 —148 —244 
—12 —48 —108 —192 
—24 —72 —144 —240 
—40 —100 —184 
—60 —132 —228 
—84 —168 
—112 —208 
—144 
—180 


We then have the following algorithm. 


(i=3)t 
n—-1 


1. Find the collocation points x; = cos 
1, and evaluate f(zx;); 


,forj =1,...,n—- 


2. Use the recurrence (1.3) to evaluate T;(a;), for €=0,...,n—2 
and 7 =1,...,n—-—1; 

3. Solve equations (10.17), in reverse order, to determine the 
coefficients Yyn—2,---,; Y1,; Yo one by one. 


EXAMPLE 10.1: Taking the differential equation 


2 


da?” 


(x)+6|2]=0, u(+1) =0, (10.18) 


whose known solution is u(#) = 1 — 2? |z|, and applying the above method with 
n = 10, we get the results in Table 10.1, where we show the values of the exact 
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and approximate solutions at the collocation points. (Both solutions are necessarily 
even.) 


Table 10.1: Solution of (10.18) by collocation, n = 10 


u(x) | Un(z,) 


10.2.2 Error of the collocation method 


We may analyse the error of the preceding collocation algorithms by the use 
of backward error analysis. 


Here we shall look only at the first of the two alternative representations 
of u(x). Let u(a) denote the true solution to the problem (10.3), let 


Un(X) = S~ ChT (2) 
k=0 


be the approximate solution obtained by collocation and let f,(a) be the 
second derivative of u,(x). Then up(x) is itself the exact solution to a similar 


problem 
2 


Sytin( 2) =fr(x), Un(—l) =a, wUn(4+1)=b. (10.19) 


Since equations (10.3) and (10.19) are both linear, and have the same bound- 
ary conditions, the error e(#) := u(x) — un(x) must be the solution to the 
homogeneous boundary-value problem 


els) =of(x), e(—1) =e(4+1) =0, (10.20) 


where 


Of (x) = f(a) — fn(2). (10.21) 


We can write down the solution of (10.20) in integral form 


e(a) = a Gla, 65) a6 (10.22) 
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where G(x, €) is the Green’s function for this problem 


_f -40-2)1+8, €<z, 
G(,§) ={ A ee), E> 2. 


Equation (10.22) may be used to derive bounds on the error e(x) from various 
possible norms of the difference 5 f(a). In particular: 


(10.23) 


le(x)| < 51-27) lI5f loo (10.24a) 
1 2 

le(x)| < Ta — @°)|l5fll. » (10.24b) 

le(x)| < 3(1— 2”) [lo fll, - (10.24c) 


We know that u,p(az) is a polynomial of degree n in x, so that its sec- 
ond derivative f,,(a) must be a polynomial of degree n — 2. The collocation 
equations (10.10a), however, tell us that 


fala) = opun( ay) = F(3) 


so that f,(a) coincides with f(x) at the n—1 points {x1,...,¢%,-1}. Therefore 
fn(x) must be the unique (n — 2)nd degree polynomial interpolating f(x) at 
the zeros of T,—1(x), and ||df|| must be the corresponding interpolation error. 


Now we may apply one of the standard formulae for interpolation error, 
for instance (Davis 1961, Chapter 3): 


e [Hermite] Assuming that f(x) has an analytic continuation into the 
complex plane, we have 


sf(0) = fle) ~ fale) = =o fp FAM (1025) 


where C' is a closed contour in the complex plane, encircling the interval 
[—1,1] but enclosing no singularities of f(z). 


e [Cauchy] Assuming alternatively that f(a) has (n—1) continuous deriva- 
tives, we have 


— _— =) ( = 
dle) = fle) - fle) = IO (10.26) 
for some real € in the interval -1 <€ <1. 
To take (10.25) a little further, suppose that the analytic continuation f(z) 


of f(x) is regular on and within the ellipse E, defined in Section 1.4.1, with 
foci at z = +1. Then we know from (1.50) that 
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for every z on E,, while |T,—1()| <1 for every x in [—1,1]. Therefore 


ea 


Applying (10.24), we deduce that the collocation solution u,,(x) converges 
exponentially to the exact solution u(x) in this case, as the number n+ 1 of 
terms and the number n — 1 of collocation points increase. 


10.2.3. Projection (tau) methods 


Just as collocation methods are seen to be related to approximation by in- 
terpolation, so there are methods that are related to approximation by least 
squares or, more generally, by projection. 


e Approximating u(x) by 


Un(x) = S~ cht p(x), 
k=0 


as in (10.9), suppose now that we select n—1 independent test functions 
{wi(x),...,%n—1(x)} and a positive weight function w(x), and solve for 
the n+ 1 coefficients {c,} the system of n + 1 linear equations 


= / wa) 4 SY SD (k= r)k(e + r)exTo(e) — fla) $ vela) de 
71 k=2  r=0 
(k—r) even 


(10.28) 
so that uy satisfies the boundary conditions and the residual 


d2 
stin(2) — fe) 


is orthogonal to each of the n — 1 test functions w(x), ...%n—1(x) with 
respect to the weight w(z). 
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If we take We(x) = Tr_1(x) and w(x) = 2(1—27)-1/?, this is equivalent 
to saying that the residual may be represented in the form 


a = 
qua tine) = F(a) = a Tr-1Tk-1(2), 


for some sequence of undetermined coefficients {7}. The method is for 
this reason often referred to as the tau method! (Ortiz 1969), although 
differing slightly from Lanczos’s original tau method (see Section 10.3 
below), in which the approximation u,,(a) was represented simply as a 


sum of powers of x 
n 
Un(x) = ) apa®. 
k=0 


In our case, we can use the orthogonality relations (4.9), (4.11) to reduce 
the first n — 1 of these equations to 


. 2 op Taye) 
(b= Ok(k+ Ocg== {| 222 
a ea -1 Vi-2? 


(k—£) even 


dz, ¢€=0,...,n—-2. 


(10.29) 


The similarities between equations (10.29) and (10.13) are no coinci- 
dence. In fact, the right-hand sides of (10.13) are just what we obtain 
when we apply the Gauss—Chebyshev quadrature rule 1 of Theorem 8.4 
(page 183) to the integrals on the right-hand sides of (10.29). 


If we use this rule for evaluating the integrals, therefore, we get precisely 
the same algorithm as in the collocation method; in many contexts we 
may, however, have a better option of evaluating the integrals more 
accurately—or even exactly. 


If we use the alternative approximation 


1-2 genteel 
a 
2 Qe 


n—-2 
Un(a) = (1 — 2) De, ynU, (x) + 
k=0 
as in (10.14), we are similarly led to the equations 


n-2 lt 
2 Te(x) f (x) 
0? — 3k? — 6k —4)y, = — ———— dr, @=0,...,n-2, 
Me Ri fea a Le? 


(k—£) even 


and the same final remarks apply. 


Compare the tau method for approximating rational functions described in Section 3.6. 
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EXAMPLE 10.2: Taking same differential equation (10.18) as previously, and now 
applying the above projection method with n = 10, we get the results in Table 10.2, 
where for convenience we show the values of the exact and approximate solutions 
at same points as in Table 10.1. It will be seen that the results are slightly more 
accurate. 


Table 10.2: Solution of (10.18) by projection, n = 10 


w(aj) | tn (as) 


10.2.4 Error of the preceding projection method 


We may carry out a backward error analysis just as we did for the collocation 
method in Section 10.2.2. 


As before, let u(a) denote the true solution to the problem (10.3) and un (x) 
the approximate solution, let f,(z) = d?un(x)/ da? and df(x) = f(x)—f,(z). 
Then the error bounds (10.24) still apply. 


Assume now that the integrals in (10.29) are evaluated exactly. The func- 
tion f,(a) will again be a polynomial of degree n — 2 but this time, instead 
of interpolating f(x) at collocation points, it is determined by the integral 
relations 

* Te(x)d f(x) 
-1 Vv 1 om x? 
In other words, f,(a) is a weighted least-squares approximation to f(x) with 
respect to the weight w(a) = (1 — 2?)~1/?; that is to say, it is the truncated 
Chebyshev series expansion 


dz=0, €=0,...,n—2. (10.31) 


ful) = Sy deTe(x) 
k=0 


of f(x). 
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Then, for instance, applying the results of Section 5.7, we can say that if 
the analytic continuation f(z) of f(a) is regular on and within the ellipse E, 


then 
M 


SFI = |F@) -— Sr-2@| S$ Gago (10.32) 


so that again 
|df(x)| = O(r~") asn > cw, (10.33) 


and the projection solution converges as n increases at the same rate as the 
collocation solution based on the zeros of T;,_1. 


10.3. The original Lanczos tau (7) method 


The original ‘tau method’ for ordinary differential equations as described by 
Lanczos (1938) approximated the unknown function by an ordinary polyno- 
mial rather than by a truncated Chebyshev expansion — Chebyshev poly- 
nomials made their appearance only in the residual. We illustrate this by a 
simple example. 


EXAMPLE 10.3: Consider the equation 
& + 4ru = 0 (10.34) 


on the interval —1 < x < 1, with the condition u(0) = 1, to which the solution is 


If we try the approximation 
ug(x) = 1+ aia + aga” + agax® + aaa* + asx” + a62°, (10.35) 
which satisfies the given condition, we come up with the residual 


aus + drug = a1 + (4+ 2a2)a + (4a1 + 3a3)x? + (4a2 + 404)" 4 


+ (4a3 + 5as)a* + (4a4 + 6ae)x” + 4asx® + daca’. 


The conventional technique (Frobenius method) for dealing with this residual would 
be to ignore the last two terms (the highest powers of x), and to equate the remaining 
terms to zero. This gives 


ag = —2, a4 = 2, a6 = a (10.36) 


with all the odd-order coefficients vanishing. 


Lanczos’s approach in the same case would have been to equate the residual to 


T6l6 (x) + 77T7 (x) : 
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This gives tT. = 0, and the odd-order coefficients again vanishing, while 


4 264 208 64 
ss - =p OE 10. 
BES agg? oA ag h See ag goed 


Thus the conventional approach gives the approximate solution 


ue(x) = 1 — 2x? + 2x4 sx° (10.38) 
while Lanczos’s method gives 


26427 — 208x7 + 642° 


35 (10.39) 


ue(x) =1— 


The improvement is clear—compare Figures 10.1 and 10.2. 


Figure 10.1: Power series solution (10.38) compared with true solution on 
ee 1] 


= ate 


Figure 10.2: Lanczos tau solution (10.39) compared with true solution on 
[aL 1] 
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10.4 <A more general linear equation 


The methods used to attack the simple equation of Section 10.2 may be applied 
with little alteration to the general linear two-point boundary-value problem 


2 
d 
qu tle) + a(x) Fula) + r(a)u(z) = f(z), u(-1)=a, u(t+l)=6, 
(10.40) 
where q(x) and r(a) are given continuous functions of x. 


Approximating u(x) again by the finite sum (10.9) 


Un(2) = S~ cxTi(2), 
k=0 


using the formula (10.5) 


k-2 


Tee) = YO (k= r)k(k + r)T, (x), (k > 2), 
r=0 
(k—r) even 


d2 
da? 


for an T;,,(a) and the formula (2.49) 
r k-1 
— = > 
5 (2) , QkT,(x), (k > 1) 
(k—r) odd 


for <LT;,(x), we get linear equations similar to (10.6) but with the first equa- 
tion (10.6a) replaced by 


n 


k-2 
! 
So SS (ka r)k(K + r)ceT p(x) + 
k=2 r=0 
(k—r) even 


n 


k-1 
+ q(x) S- S~ QhepT,-(a) + 


k=1 r=0 
(k—r) odd 


+ r(x) S~ xT (2) = f(x). (10.41) 
k=0 


10.4.1 Collocation method 


If we substitute « = 71, ..., © = Lp_1 in (10.41), we again get a system of 
linear equations for the coefficients co, ..., Cn, which we can go on to solve 
directly. 
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If q(x) and r(a) are polynomials in x, and therefore can be expressed as 
sums of Chebyshev polynomials, we can go on to use the multiplication for- 
mula (2.38) or that quoted in Problem 4 of Chapter 2 to reduce the products 
q(x)T; (a) and r(x)T;,(a#) in (10.41) to simple sums of Chebyshev polynomi- 
als T(x). We can then use discrete orthogonality as before to simplify the 
equations to some extent. 


Whether this is possible or not, however, collocation methods for linear 
problems are straightforward to apply. 


It should be noted that the error analysis in Section 10.2.2 does not extend 
to this general case. The reason why it breaks down is that, where previously 
we could say that f,(a) was an interpolating polynomial of degree n — 2, we 
now have the more complicated expression 

d? d 
fale) = pial) + ae) un(t) + 1(0)Un (2). 


We can therefore no longer appeal to formulae for polynomial interpolation 
error. 


10.4.2 Projection method 


If g(a) and r(x) are polynomials in x, and we can therefore proceed as in the 
collocation method to reduce q(x)T;-(a) and r(x)T;(x) in (10.41) to simple 
sums of Chebyshev polynomials T;,(x), then we can use integral orthogonality 
relations (multiplying by 2(1— x?)~2T;(x) and integrating, for 2= 0,...,n— 
2) to derive a set of linear equations to solve for the coefficients {c;,}. 


In more general circumstances, however, we may need either to approxi- 
mate q(x) and r(x) by polynomials or to estimate the integrals 


* Ti(x)q(x)Tr (2) * Ti(a)r(a)Te (a) 
dx dx 
=p V1 — x? , ease V1l— 2x? 


numerically. 


10.5 Pseudospectral methods — another form of collocation 


In the collocation method we discussed earlier in Section 10.2.1, we approx- 
imated u(x) by the truncated Chebyshev expansion u,,(a) of (10.9), an nth 
degree polynomial. Instead of representing this polynomial by its n+ 1 Cheb- 
yshev coefficients, suppose now that we represent it by its values at the two 
boundary points (a and z,) and at n— 1 internal collocation points (1, 

.., Ln—1); these n + 1 values are exactly sufficient to define the polynomial 
uniquely. According to some writers, the coefficients yield a spectral and the 
values a pseudospectral representation of the polynomial. To make use of such 
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a representation, we need formulae for the derivatives of such a polynomial in 
terms of these values. 


10.5.1 Differentiation matrices 


Suppose that we know the values of any nth degree polynomial p(x) at n+ 1 
points Xo,...,£,. Then these values determine the polynomial uniquely, and 
so determine the values of the derivatives p'(x) = dp(x)/da at the same 
n+ 1 points. Each such derivative can, in fact, be expressed as a fixed linear 
combination of the given function values, and the whole relationship written 
in matrix form: 


p' (0) doo ++: don p(xo) 
Ball 04) Be i (10.42) 
p' (tn) dno acts dn.n D(tn) 
We shall call D = {dj} a differentiation matriz. 


Suppose now that the points x; are the n+1 zeros of some (n+1)st degree 
polynomial P,,41(2). 


If for k = 0,...,n we let pr(x) = Pr4i(x)/(e% — xx), which is an nth 
degree polynomial since x, is a zero of P,41, then we can show without much 
difficulty that 


pr(tz) = Pos(&r) 
1 
P(e) = 5 Pnti(ee) 


Priltj) 
py(tj) = SS, fF Fk. 
Lj — Xk 


From this we can deduce (by setting p(x) = p,(x) in (10.42)) that the kth 
column of the differentiation matrix D must have elements 


1 Priya (@e) 
dik = IP" P(e)’ (10.43a) 
ip So ay (10.43b) 


(25 — &e) Phi (te) ’ 


Notice that if p(ao) = p(a1) =--: = p(a@»,) = 1 then we must have p(x) = 1 
and p’(x) = 0. It follows that each row of the matrix D must sum to zero, so 
that the matrix is singular, although it may not be easy to see this directly 
by looking at its elements. 
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Not only can we use the relationship 
p'(xo) P(x) 
; [=D]: 
p'(&n) P(Zn) 
to connect the first derivatives with the function values, but we can repeat the 
process (since p’ is an (n — 1)st degree polynomial, which may be regarded as 
an nth degree polynomial with zero leading coefficient), to give us a similar 
relationship for the second derivatives, 
p" (xo) p(xo) 
: =D? : 
p' (Zn) P(&n) 


and so on. 


10.5.2 Differentiation matrix for Chebyshev points 


In particular, suppose that the n+ 1 points are the points y;, = cos An which 
are the zeros of the polynomial P,+41(a) = (1 — x?)Un—1(x) and the extrema 
n [—1, 1] of T,(#). Making the usual substitution x = cos 0 gives us 


Prii(x) = sin@ sin nd. 
Differentiating with respect to x, we then have 


P!,, (2) - _ cos sinné+nsin@ cosné 


10.44 
sin 0 ( ) 
and 
z cos? @ sinn@ —nsin@ cos@ cosné + (1+ n?)sin? 6 sin nd 
LO) 
sin” 6 
(10.45) 


However, we know that if 6, = En then sinn@, = 0 and cos nO, = (—1)*. 
Therefore (10.44) gives us 
Prsilye) = 4 —2n, k=0, (10.46) 


and (10.45) gives 


Yk 
(-1)* ; O0<k<n, 
1— yz 
1+ 2n? 
Prsiye) = 4 —2n = = k=0, (10.47) 
2 
2(—1)" 1+ 2n a 
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(In each case, the values for k = 0 and k = n are obtained by proceeding 
carefully to the limit as x — 1 and x — —1, respectively.) 


Substituting (10.46) and (10.47) in (10.43a) and (10.43b) gives the follow- 
ing as elements of the differentiation matrix D: 


1 
2(14+2n?) 
s(1-+2n?) 2 

di.<= a1 
71 = Vi ag a 

4 1 _ 1 
2 1— ye y2—- Y1 
“4 (=1)"" (-1)"-? 
2 I ee Yn-1 Yn-1 — Yl 
_4)nr-1 
1+y1 

For instance 
n=1 (ye = 1, =a) 
D — 


NIP le 


Yk 
dye = —4 », O0<k<n, 
dnn = —@(1 + 2n?), 
—1)* 
dy.o = —+ ( , O<k<n, 
me Al Yi 
=]; n—-k 
dn k _ah ) , O<k<n, 
1+ Yk 
dno = —3(-1)" 
(10.48) 
(-1)""* 1(_4y)n 
25 ae 5( 1) 
(=1)"-? ve Ge? 9 a 
Y1 — Yn-1 2 14+y1 
(-1jy"? rd ee 2 
Y2 — Yn-1 2 1l+ye 
Pn ne a ee 
tg Pai 
1 4 2 
14+ Yn-1 aire) 
(10.49) 
D2 = 0 0 <i 
0 0 /’ 
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n=2 (yx = 1, 0, -1) 


3 1 
z 2 3 1.58 
D= , O -3 |, D?=[ 1 -2 1 |; 
be 94 
-$ 2-3 


19 ag 4 22 16 _28 2 _8 

6 3 2 3 3 3 3 

1 -2 1 1 16% 2i6- 8. 2% 
D= 3 3 D2 = 3 3 3 3 : 
aie yp A ap |? a2. Be - ee 

3 3 3 3 3 3 

A 2m, vat cei 28 BO) 5 De? 16 

2 3 6 3 3 3 3 


n=4 (yr =1, 1/V2, 0, —1/V2, —1) (Fornberg 1996, p.164)? 


4 4-2/2 2 4427/2 4 
14+1//2 -1/V2 -V2 if/Vv2 -14+1//2 
D= 25 v2 0 —V2 5 
1-1/y2 -1//2 V2 o1f/v2- -1-1/v2 
—4 4-2/2 -2 4422 -# 
17 —20-6/2 18 -20+6,/2 5 
5+3/V/2 —14 6 me) 5-3/2 
D? = —1 4 —6 4 —1 
5 = 3//2 —2 6 —14 5+3/2 
5 =20+6/2. 18 —20—6V2 17 


Notice that the matrices D and D? are singular in each case, as expected. 


10.5.3 Collocation using differentiation matrices 


We return first to the simple example of (10.3) 


ula) = f(x), u(-l)=a, u(+1)=6), (10.50) 


?Fornberg numbers the nodes in the direction of increasing y,—we have numbered them 
in order of increasing k and so of decreasing yz. 
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and suppose that the collocation points {x,;} are chosen so that zp = +1 and 
Ln = —1 (as will be the case if they are the Chebyshev points {y;} used in 
Section 10.5.2.) 


We know that 
2 n 


d 
dx? Un (aj) = YD"); (we). 
k=0 
The collocation equations thus become 


n 


S°(D? )jkUn(@e) = f(zj), j=1,...,n-1, Un(@n) =a, Un(to) = 0. 
k=0 
(10.51) 


Partition the matrices D and D? as follows, by cutting off the first and 
last rows and columns: 


(10.52) 


and let u and f denote the vectors 


Un(21) f(x1) 


Un (Xn—1) f(#n—-1) 

The collocation equations (10.51) can then be written in matrix notation as 
Un (xo)e? 4 But Un(tn)e? = f, 

or 

E@u =f — be? — ae, (10.53) 


since the values of u,(xo) and un(xp) are given by the boundary conditions. 
We have only to solve (10.53) to obtain the remaining values of un (x). 


In order to obtain the corresponding Chebyshev coefficients, we can then 
make use of discrete orthogonality relationships, as in Section 6.3.2. 


Similarly, in the case of the more general equation (10.40) 


2 
Sula) + al) —u(2) +r(x)u(xz) = f(x), u(-l)=a, u(+1)=), 
(10.54) 
if we let Q and R denote diagonal matrices with elements q(x;,) and r(x) 
(k =1,...,n—1), the collocation equations can be written as 


(B°) + QE+ R)u =f — b(ef” + Qeo) — a(el?) + Qeo). (10.55) 
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10.6 Nonlinear equations 


We mention briefly that the techniques discussed in the preceding Sections 
can sometimes be extended to include nonlinear equations. 


To take one simple example, using pseudospectral methods and following 
the principles of (10.51) and (10.52), the problem 


Sula) = f(u(x)), u(-1) = u(+1) = 0, (10.56) 


where f(u) is an arbitrary function of u, gives rise to the system of equations 


S-(D?);,6Un(ae) = f(un(z5)), j=l,...,.n-1, Un(n) oe Un(Xo) = 0, 
k=0 
(10.57) 
or, in matrix terms, 
Eu = £(u), (10.58) 


where f(u) denotes the vector with elements {f(tun(z;))}. 


Equations (10.58) may or may not have a unique solution. If they do, or 
if we can identify the solution we require, then we may be able to approach 
it by an iterative procedure. For instance: 


simple iteration Assume that we have a good guess u at the required 
solution of (10.58). Then we can generate the iterates u), u®), and so 
on, by solving successive sets of linear equations 


BQu® =f(u'-)), k=1,2,.... (10.59) 


Newton iteration Provided that f(w) is differentiable, let f’(u) denote the 
diagonal matrix with elements {f’(u,(z;))}, and again assume that we 
have a good guess u) at the required solution. Then generate successive 
corrections (u“) — u()), (a?) — u™), and so on, by solving successive 
sets of linear equations 


(Eo zi f(u*-)) (ul) — wD) = g(u@-D) — BA yE-D, 


an ey ae (10.60) 


There is no general guarantee that either iteration (10.59) or (10.60) will 
converge to a solution — this needs to be studied on a case-by-case basis. If 
both converge, however, then the Newton iteration is generally to be preferred, 
since its rate of convergence is ultimately quadratic. 


252 CHAPTER 10: SOLUTION OF ORDINARY DIFFERENTIAL EQUATIONS 


10.7 Eigenvalue problems 


Similar techniques to those of the preceding sections may be applied to eigen- 
value problems in ordinary differential equations. One would not, of course, 
think of applying them to the simplest such problem 


2 
u(2) +ru(z) =0, u(+tl1)=0, (10.61) 
since its solutions 
u(z) =singkn(x@+1), A=($kr)?, k=1,2,..., (10.62) 


can be written down analytically. This problem nevertheless has its uses as a 
test case. 


Less trivial is the slightly more general problem 
2 


dx? 


where g(a) is some prescribed function of x. 


u(x) + q(a)u(x) + Au(xz) = 0, u(+1) = 0, (10.63) 


10.7.1 Collocation methods 


If we approximate u(x) in the form (10.9) 


e / 
TACs ee COR (10.64) 
k=0 
and select the zeros (10.11) of T,-1(2) 
UAe -s 
; = cos ——+— =1,...,.n-1 10. 
j= COS ——, J yee, n—1, (10.65) 


as collocation points, then the collocation equations for (10.63) become 
nba? 
S- (k—r)k(k + r)cxT(a3) + 
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Using discrete orthogonality wherever possible, as before, (10.66a) gives 
us the equations 


n 2 n-1 1 ; 
So (k-Q) K(k + Dew + a S© (aj) To( 25) Te (#5 )ew + 
a j=1 k=0 
al ACe = 0, 
£=0,...,n—2. (10.67) 


tr =—- >> ch, (10.68a) 
k=0 
(n—k) even 
ae 
C@1=- >> & (10.68b) 
k=0 
(n—k) odd 


to reduce (10.67) to the form of a standard matrix eigenvalue equation of the 
form 


Co Co 
Cl C1 
A =i ; (10.69) 
Cn—2 Cn—2 


which we may solve by standard algebraic techniques. 


This will yield n — 1 matrix eigenvalues {\)}, with corresponding eigen- 
vectors {ce}. These eigenvalues should approximate the n — 1 dominant 
eigenvalues of (10.63), with 


S~ nel? Th (2) 
k=0 


approximating the eigenfunction u(x) corresponding to the eigenvalue ap- 
proximated by A“), (Equations (10.68) are used to obtain the coefficients cl) 
(9) ) 


We illustrate this with the trivial example (10.61) in which g(x) = 0, when 
(10.67) becomes 


S (k=O klk + Oc + Ace = 0, 


k=42 
(k—£) even 
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Table 10.3: Eigenvalues of (10.70) and differential equation (10.61) 


Matrix O.D.E. 
2.4674 2.4674 
9.8696 9.8696 

22.2069 22.2066 


39.6873 39.4784 


62.5951 61.6850 
119.0980 88.8624 
178.5341 120.9026 

1991.3451 157.9136 
3034.1964 199.8595 


Results for n = 10 are shown in Table 10.3. 


Note, incidentally, that if g(a) is an even function of x, so that 
va q(x;)Te(x;)Th(2;) =0, k—Il even, 


we can separate the odd-order coefficients c, from the even-order ones, thus 
halving the dimensions of the matrices that we have to deal with. 


10.7.2 Collocation using the differentiation matrix 
The methods of Section 10.5.3 may be applied equally well to eigenvalue 
problems. For instance, the problem posed in (10.63) 


Sula) + q(x)u(x) + Au(z) = 0, u(+1) =0, 


is the same as (10.54) 
d? d 
sa Ul(2) + (a) ula) + r(a)u(a) = f(z), u-l)=a, u(+1) =), 
da? dx 
with r(z) =, f(z) =O anda=b=0. 
Thus equation (10.55) becomes 


(BE?) + QE+ A)u=0, (10.71) 


where 
q(x1) 


& 
I 


q(%n—1) 
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where E and E®) are defined as in (10.52) on page 250 and where 


Un(21) 


Un(Ln—1) 


Equation (10.71) is another standard matrix eigenvalue equation. 


The test case (10.61) gives the simple eigenvalue problem 
E?u+Au=0. (10.72) 


We may use this to illustrate the accuracy of the computed eigenvalues. It 
will be seen from the last example in Section 10.5.2 that the matrix E®) will 
not always be symmetric, so that it could conceivably have some complex 
pairs of eigenvalues. However, Gottlieb & Lustman (1983) have shown that 
its eigenvalues are all real and negative in the case where the collocation 
points are taken as Chebyshev points (although not in the case where they 
are equally spaced). Whether the same is true for the matrix E?) + QE 
depends on the size of the function q(z). 


Table 10.4: Eigenvalues of differentiation matrices and differential equation, 
Chebyshev abscissae 


Matrix O.D.E. 
—2.4668 —2.4674 2.4674 
—9.6000 —9.8696 9.8696 


—31.1332 | —22.2060 22.2066 


—40.0000 | —39.5216 39.4784 
—60.7856 61.6850 

—97.9574 88.8624 

—110.8390 120.9026 

—486.2513 157.9136 

—503.3019 199.8595 


We show in Table 10.4 the computed eigenvalues (which are indeed all 
real and negative) of the (n — 1) x (n — 1) matrix E®) corresponding to 
the Chebyshev collocation points {y,}, for the cases n = 5 and n = 10, 
together with the dominant (smallest) eigenvalues of (10.61). We see that 
respectively three and five of these eigenvalues are quite closely approximated. 
For general values of n, in fact, the lowest |2n/]| eigenvalues are computed 
very accurately. 
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Table 10.5: Eigenvalues of differentiation matrices and differential equation, 
evenly spaced abscissae 


Matrix O.D.E. 
—2.4803 —2.4674 2.4674 
—11.1871 —9.8715 9.8696 


—15.7488 —22.3049 22.2066 


—17.4587 —36.3672 39.4784 


—48.5199 61.6850 
(—57.6718 + 88.8624 
+ 45.9830 i) 120.9026 
(—58.2899 + 157.9136 
+ 62.5821 i) 199.8595 


Table 10.5 displays the corresponding results when the Chebyshev points 
are replaced by points evenly spaced through the interval [—1,1]. We see 
that not only are the lower eigenvalues less accurately computed, but higher 
eigenvalues can even occur in complex pairs. 


The same phenomena are illustrated for n = 40 by Fornberg (1996, Fig- 
ure 4.4-2), 


10.8 Differential equations in one space and one time dimension 


A general discussion of the application of Chebyshev polynomials to the solu- 
tion of partial differential equations will be found in Chapter 11. A particular 
class of partial differential equations does, however, fall naturally within the 
scope of the present chapter—namely equations in two independent variables, 
the first of which (t, say) represents time and the second (x, say) runs over a 
finite fixed interval (which as usual we shall take to be the interval [—1, 1]). 
We may then try representing the solution at any fixed instant ¢ of time in 
terms of Chebyshev polynomials in x. 


For a specific example, we may consider the heat conduction equation 


2 
<ult,2) = a(x) -yult,2), t>0, -l<a<l, (10.73a) 

where q(x) > 0, with the boundary conditions 
u(t, -1) = u(t, +1) =0, t>0, (10.73b) 


and the initial conditions 


u(0,z) =uo(z), -l<a<l. (10.73c) 
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10.8.1 Collocation methods 


We try approximating u(t, x) in the form 


un(t,2) = S~ ex(t)Te(2); (10.74) 


k=0 


and again select the zeros (10.11) of T,-1(2) 


_l 
©; = cos e aug ee eae (10.75) 
as collocation points. The collocation equations for (10.73) become 
n | n ae 
Ly ap ck OTe (25) = g(a) ¥> So (k= r)k(R + r)cu(t)Tr (a5), 
k=0 k=2 r=0 
(k—r) even 
j=l,...,n—-1, (10.76a) 
S~ (-1)Fex(t) = 0, (10.76b) 
k=0 
u / 
yn = 0 (10.76c) 
k=0 
Using discrete orthogonality as before, equations (10.76a) give 
n k-2 = 
d ete ! (I ir 
Ha) = Lele tr ae Te(arj)Te(a)eu(t), 
"(Keven = 

€=0,...,n—2. (10.77) 
We again make the substitutions (10.68) for c,-1(t) and c,(t), giving us a 

system of linear differential equations for co(t), ..., Cn—2(t), of the form 

co(t) co(t) 
d{ e(t) cx(t) 
ates =A : 10.78 
Cn—2(t) Cn—2(t) 


The matrix A in (10.78) is not the same as the one appearing in (10.69); 
however, if we can find its eigenvectors {ec} and eigenvalues {\Y)}, then 
we can write down the general solution of (10.78) as a linear combination of 
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terms {exp A“ ¢ ce}, and hence find the solution corresponding to the given 
initial conditions (10.73c). 


We shall not discuss here the question of how well, if at all, the solution 
to this system of differential equations approximates the solution to the orig- 
inal partial differential equation (10.73). In particular, we shall not examine 
the possibility that some \% have positive real parts, in which case the ap- 
proximate solution would diverge exponentially with time and therefore be 
unstable and completely useless. 


10.8.2 Collocation using the differentiation matrix 
Once more, we have an alternative approach by way of differentiation matrices. 
The heat conduction problem (10.73) 


—u(t,x) = g(a) pull x), u(t,+1)=0, 


is another that can be derived from (10.54) 


by replacing u(x) and <4 by u(t,x) and Z, and setting g(#) = r(x) = 0, 
a=b=0 and 


1 0 
= —~— ull, Zz). 
(2) res ap ult =) 
In place of equation (10.55) we thus find the system of differential equations 
d 
—u(t) = QE?) u(t), (10.79) 


where 


Utn—-1) 
where E() is defined as in (10.52) on page 250 and where 
Un(t, v1) 
u(t) = 
Un (t, Ln—1) 
As in the case of (10.78), we may write down the solution of this sys- 
tem of equations as a linear combination of terms {exp \\)¢ u}, where the 


matrix QE) has eigenvectors {u\)} and eigenvalues {\\)}. In the spe- 
cial case where q(x) is constant, q(x) = q > 0 so that QE?) = qE®), we 
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recall from Section 10.7.2 that the eigenvalues will all be real and negative 
(Gottlieb & Lustman 1983), provided that collocation is at Chebyshev points; 
consequently all of these terms will decay exponentially with time and the ap- 
proximate solution is stable. 


10.9 Problems for Chapter 10 


1. Show that the two collocation algorithms of Section 10.2.1 should lead 
to exactly the same result for any given value of n—likewise the two 
projection algorithms of Section 10.2.3. 


2. (a) Consider the problem 
(1—a)y’=1o0n [0,1], y(0) =0. 


Obtain polynomial approximations y,(@) = co +c, +++: + cnx” 
to y(x), of degrees n = 1, 2,3, by including a term TT*(x) on the 
right-hand side of the equation. What is the true solution? Plot 
the errors in each case. 


(b) Consider the slightly modified problem 


(1—2)y' =1 on [0,3], y(0) =0. 


How do we apply the tau method to this problem? Repeat the 
exercise of the previous example using TT,‘ (42/3). 
(c) Repeat the exercises again for the intervals [—1, 1] (using TT, («)) 


and [—2, 3] (using rT;,(4a/3)). What effect does extending the 


interval have on the approximate solution and the size of the error? 


3. Where necessary changing the independent variable so that the interval 
becomes [—1, 1], formulate and solve a (first-order) differentiation matrix 
approximation to one of the parts of Problem 2. 


4. Obtain a numerical solution to the differential equation (10.18) by the 
standard finite-difference method 


(uj—-1 — 2u; + uj41)/h? + 6 |xj| = 0, A =U — 0; 


where h = 1/n, x; =j/n (|j| <n) and u; approximates u(z;). 


For n = 10, say, how does the solution compare with the Chebyshev 
solutions in Tables 10.1 and 10.2? 


5. Verify formulae (10.44) and (10.45). 


6. Justify the limiting values given in (10.46) and (10.47). 
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. Investigate the application of Chebyshev and conventional finite-difference 


methods to the solution of the differential equation 


(1+ 2522)?“ u(a) = 50(75x2? —1)u(x), u(+1)= 4 


2 26? 


whose exact solution is the function 1/(1 + 252?) used to illustrate the 
Runge phenomenon in Section 6.1. 


. Investigate similarly the non-linear equation 


d? 1 


qe) + eae mE =0, u(+1)=0. 


. Verify the eigenvalues quoted in Tables 10.4, 10.5. 


CHAPTER 11 


Chebyshev and Spectral Methods for Partial Differential 
Equations 


11.1 Introduction 


Chebyshev polynomial applications to partial differential equations (PDEs) 
Eu =0 on a domain S, (11.1a) 
subject to boundary conditions 
Bu =0 on 0S, (11.1b) 


where OS is the boundary of the domain S$, are a natural progression of 
the work of Lanczos (1938) and Clenshaw (1957) on ordinary differential 
equations. However, the first formal publications in the topic of PDEs ap- 
pear to be those of Elliott (1961), Mason (1965, 1967) and Fox & Parker 
(1968) in the 1960s, where some of the fundamental ideas for extending one- 
dimensional techniques to multi-dimensional forms and domains were first 
developed. Then in the 1970s, Kreiss & Oliger (1972) and Gottlieb & Orszag 
(1977) led the way to the strong development of so-called pseudo-spectral 
methods, which exploit the fast Fourier transform of Cooley & Tukey (1965), 
the intrinsic rapid convergence of Chebyshev methods, and the simplicity of 
differentiation matrices with nodal bases. 


Another important early contribution was the expository paper of Fin- 
layson & Scriven (1966), who set the new methods of the 1960s in the context 
of the established “method of weighted residuals” (MWR) and classified them 
formally into the categories of Galerkin, collocation, and least squares meth- 
ods, as well as into the categories of boundary, interior and mixed methods. 


Let us first clarify some of this nomenclature, as well as looking at early 
and basic approximation methods. We assume that the solution of (11.1a), 
(11.1b) is to be approximated in the form 


Ud Un = f(Ln) (11.2) 
where 


i S- cede (11.3) 
kat 


is a linear combination of an appropriate basis of functions {d,} of the inde- 
pendent variables (a and y, say) of the problem and where f is a quasi-linear 
function 

f(L=AL+B, (11.4) 


where A, B are specified functions (of x and y). 
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11.2 Interior, boundary and mixed methods 


11.2.1 Interior methods 


An interior method is one in which the approximation (11.2) exactly satisfies 
the boundary conditions (11.1b) for all choices of coefficients {c;}. This is 
typically achieved by choosing each basis function ¢; appropriately. If Bu in 
(11.1b) is identically u, so that we have the homogeneous Dirichlet condition 


u=0o0n OS, (11.5) 


then we might well use the identity function for f, and choose a basis for 
which every ¢; vanishes on OS. For example, if S is the square domain with 
boundary 


OS:x2=0, cx=1, y=0, y=1. (11.6) 
then one possibility would be to choose 
op = Oi =sinine sin jax (11.7) 
with 
k=i+n(j-1) 
and 
Ck = Aj; 
say, so that the single index k = 1,...,n? counts row by row through the 
array of n? basis functions corresponding to the indices i = 1,...,n and 
j =1,...,n. In practice we might in this case change notation from ¢; to 


®;; and from uy2, Ly2 to Unn, Lnn, setting 
u& Unn = f (Lan) 


where bist 
Enn = >, >, aij ¥ij (x,y). (11.8) 
i=1 j=1 
It only remains to solve the interior problem (11.1a). 
There is a generalisation of the above method, that is sometimes applicable 
to the general Dirichlet boundary conditions 
u= B(x, y) (11.9) 
on the boundary 
LP: A(z,y) = 0, 


where we know a formula A = 0 for the algebraic equation of I, as well as a 
formula B = 0 for the boundary data. Then we may choose 


U~ Unn = f(Lnn) = A(x, y)Lnn + Ba, y), (11.10) 
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OSs, 
P 


A \ 


Figure 11.1: 


which automatically satisfies (11.5), whatever we take for Ln,. See Mason 
(1967) for a successful application and early discussion of such techniques. 


In the discussion that follows we assume unless otherwise stated that f is 
the identity, so that u,, and L,, are the same function. 


11.2.2. Boundary methods 


A boundary method is one in which the approximation (11.2) exactly satisfies 
the PDE (11.1a) for all choices of coefficients {c;}. If the PDE is linear, for 
example, then this is achieved by ensuring that every basis function ¢, is a 
particular solution of (11.la). This method is often termed the “method of 
particular solutions” and has a long history — see for example Vekua (1967) — 
and indeed the classical method of separation of variables for PDEs is typically 
of this nature. It remains to satisfy the boundary conditions approximately 
by suitable choice of coefficients {c;}. 


For example, consider Laplace’s equation in (r,@) coordinates: 


20?u Ou Ou 


in the disk S: r < 1, together with 
u= g(0) (11.11b) 


on 0S’: r= 1, where g is a known 27-periodic function of the orientation 0 of 
a general point, P say, on the boundary (Figure 11.1). 


Then 


/ 


u~ Un(r,0) = S© [ax(r* cos(k8)) + by(r* sin(kd))] (11.12) 
k=0 


is an exact solution of (11.11a) for all {ax, by}, since r* cos(k0) and r* sin(k@) 
are particular solutions of (11.1la), which may readily be derived by separa- 
tion of variables in (11.1la) (see Problem 1). 
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Substituting (11.12) into (11.11b) gives 


/ 


u=g(0)~ = S5 [ax cos(k0) + by sin(k6)).- (11.13) 
k=0 


Clearly we require the latter trigonometric sum to approximate g(@). This 
may theoretically be achieved by choosing az, and by to be coefficients in the 
full Fourier series expansion of g(@), namely 


2 20 
ap = a) g(0) cos(k0) dO, by = oy g(0) sin(k0) dé. (11.14) 
0 0 


These integrals must be replaced by numerical approximations, which may be 
rapidly computed by the fast Fourier transform (FFT, see Section 4.7). The 
FFT computes an approximate integral transform, by “exactly” computing 
the discrete Fourier transform given by 


ax =n7! S~ g(6;)cos(k0;), be =n7' S* g(6;)sin(kO;), (11.15) 


6;=in/n (@=0,...,2n). (11.16) 


Here the periodic formulae (11.14) have been approximated by Filon’s rule, 
namely the Trapezoidal rule for trigonometric functions, which is a very ac- 
curate substitute in this case. 


Several examples of the method of particular solutions are given by Mason 
& Weber (1992), where it is shown that the method does not always converge! 
See also, however, Fox et al. (1967) and Mason (1969) where the “L-shaped 
membrane eigenvalue problem” is solved very rapidly and accurately by this 
method. 


Boundary MWR methods are important because, when they are appli- 
cable, they effectively reduce the dimension of the problem by restricting it 
to the domain boundary. In consequence such methods can be very efficient 
indeed. Moreover, because they normally incorporate precise features of the 
solution behaviour, they are often very accurate too — see Mason (1969) 
where the first L-shaped membrane eigenvalue is computed correct to 13 sig- 
nificant figures for (n =)24 basis functions. 


However, boundary MWR methods are not the only available techniques 
for in effect reducing the problem dimension. The method of fundamental 
solutions, which has been adopted prolifically by Fairweather & Karageorghis 
(1998), uses fundamental PDE solutions as a basis. These solutions typically 
have singularities at their centres, and so must be centred at points exterior 
to S. This method is closely related to the boundary integral equation (BIE) 
method and hence to the boundary element method (BEM) — for which 
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there is a huge literature (Brebbia et al. 1984, for example), and indeed the 
boundary integral equation method adopts the same fundamental solutions, 
but as weight functions in integral equations. For example, functions behaving 
like log r occur in both the method of fundamental solutions and the boundary 
integral equation method for Laplace’s equation in two dimensions. 


Both the BIE method and the BEM convert a PDE on a domain into 
an integral equation over its boundary. They consequently have the possi- 
bility for considerable improvements in efficiency and accuracy over classical 
finite element methods for the original PDE, depending on the nature of the 
geometry and other factors. 


11.2.3. Mixed methods 


A mized method is one in which both the PDE (11.la) and its boundary 
conditions (11.1b) need to be approximated. In fact this is generally the case, 
since real-life problems are usually too complicated to be treated as boundary 
or interior problems alone. Examples of such problems will be given later in 
this chapter. 


11.3 Differentiation matrices and nodal representation 


An important development, which follows a contrasting procedure to that of 
the methods above, is to seek, as initial parameters, not the coefficients c, in 
the approximation form L,, (11.3) but instead the values un(x;,y;) of Un at 
a suitable mesh of Chebyshev zeros. Derivatives can be expressed in terms of 
these uy, values also, and hence a system of (linear) algebraic equations can 
be formed for the required values of u,. It is then possible, if required, to 
recover the coefficients c, by a Chebyshev collocation procedure. 


An example of the procedure was given in Chapter 10 (Section 10.5.1) 
for ordinary differential equations (ODEs). In the case of PDEs it should be 
noted that the procedure is primarily suited to rectangular regions. 


11.4 Method of weighted residuals 


11.4.1 Continuous MWR 
The standard MWR, which we call the continuous MWR, seeks to solve an 


interior problem by finding an approximation of the form (11.2) which min- 
imises, with respect to cy, (k = 1,...,n), the expression 


(Eun, We)” = | [oun as]. (11.17) 
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where W;, is a suitable weight function (Finlayson & Scriven 1966). Here we 
assume that F is a linear partial differential operator. More specifically : 


(i) MWR is a least squares method if 
W, =w.Eun, (k=1,...,n), (11.18) 


where w is a fixed non-negative weight function. Then, on differentiating 
(11.17) with respect to cz, we obtain the simpler form 


(Eun, w.E¢,) =0, (k=1,...,n). (11.19) 


This comprises a linear system of n equations for cx. 
(ii) MWR is a Galerkin method if 


We = w.Qk. (11.20) 


Note that, in this case, we can give a zero (minimum) value to (11.17) by 
setting 
(Eun, w.dp) =0, (k=1,...,n), (11.21) 


again a system of linear equations for c,. It follows from (11.21) that 
(Eun, W.Un) = 0. (11.22) 


More generally, we can if we wish replace ¢, in (11.21) by any set of test 
functions Wz, forming a basis for uz and solve 


(Eun, we) =0, (k=1,...,n). (11.23) 


(iii) MWR is a collocation method (interpolation method) at the points 
Prius c Pe it 
Wi. — 6(Px), (11.24) 


where 6(P) is the Dirac delta function (which is infinite at the point P, van- 
ishes elsewhere and has the property that (uw, 6(P)) = u(P) for any well- 
behaved function u). Then Eu, in (11.17) will be set to zero at P,,, for every 
k. 


11.4.2. Discrete MWR — a new nomenclature 


It is also possible to define a discrete MWR, for each of the three types of 
methods listed above, by using a discrete inner product in (11.17). Commonly 
we do not wish, or are unable, to evaluate and integrate Fu,.W, over a 
continuum, in which case we may replace the integral in (11.17) by the sum 


S"(Bun)-We, (11.25) 


Sn 
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where S,, is a discrete point set representing S. 


The discrete MWR, applied to an interior problem, is based on a discrete 
inner product. It seeks an approximation of the form (11.2) which solves 


2 


Pp 
min || S/ Bun(x;)We(x;) | = (Bun, We)? | , (11.26) 
Ck 
j=l 
where x; (j = 1,...,p) are a discrete set of nodes in S, selected suitably 


from values of the vector x of independent variables, and W; are appropriate 
weights. 


(i) The discrete MWR is a discrete least-squares method if 
Wy, = wEun. (11.27) 
This is commonly adopted in practice in place of (11.18) for convenience and 


to avoid integration. 
(ii) The discrete MWR is a discrete Galerkin method if 


Wr = wok (11.28) 
or, equivalently, 
(Eun, wr) = 0. (11.29) 


Note that the PDE operator Eu, is directly orthogonal to every test function 
Wr, as well as to the approximation un, so that 


(Eun, Wun) = 0. (11.30) 


(iii) The discrete MWR is a discrete collocation method if (11.24) holds, 
where {P,} is contained within the discrete point set Sp. 


11.5 Chebyshev series and Galerkin methods 


The most basic idea in Chebyshev polynomial methods is that of expanding a 
solution in a (multiple) Chebyshev series expansion, and using the partial sum 
as an approximation. This type of approach is referred to as a spectral method 
by Gottlieb & Orszag (1977). This type of ODE/PDE method had previously, 
and still has, several other names, and it is known as (or is equivalent to) a 
Chebyshev series method, a Chebyshev—Galerkin method, and the tau method 
of Lanczos. 


Before introducing PDE methods, we consider the Lanczos tau method: 
one of the earliest Chebyshev methods for solving a linear ODE 


Ey =0 
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in the approximate form yn. 


Lanczos (1938) and Ortiz and co-workers (Ortiz 1969, Freilich & Ortiz 
1982, and many other papers) observed that, if y, is expressed in the power 
form 


Yn = bo + bya + box? +--+ + dna”, (11.31) 


then, for many important linear ODEs, Ey, can be equated to a (finite) 
polynomial with relatively few terms, of the form 


EYyn = 711 g41(@) + TeTg42(%) +--+ + 7sTo+s (x), (11.32) 


where q and s are some integers dependent on E. The method involves sub- 
stituting y, (11.31) into the perturbed equation (11.32) and equating powers 
of x from x° to 27’. The t (say) boundary conditions are also applied to yn, 
leading to a total of g+s+t+1 linear equations for bo,...,0n, T1,.--,7s. We 
see that for the equations to have one and only one solution we must normally 
have 

q¢+t=n. (11.33) 


The equations are solved by first expressing bo,...,b, in terms of 7),...,7s, 
solving s equations for the 7 values and hence determining the b values. Be- 
cause of the structure of the resulting matrix and assuming s is small com- 
pared to n, the calculation can routinely reduce to one of O(n) operations, 
and hence the method is an attractive one for suitable equations. 


The above method is called the (Lanczos) tau method - with reference to 
the introduction by Lanczos (1938) of perturbation terms, with coefficients 
T1,--+-,T7s, on the right hand side of Ey = 0 to enable the ODE to be exactly 
solved in finite form. The nice feature of this approach is that the tau values 
give a measure of the sizes of the contributions that the perturbation terms 
make to the ODE — at worst, 


|Eyn| < |ri] + |72| +--+ [7s]. (11.34) 


For some special cases, Lanczos (1957), Fox & Parker (1968), Mason (1965), 
Ortiz (1980, 1986, 1987), Khajah & Ortiz (1991) and many others were able 
to give quite useful error estimates based on the known form (11.32). 


The tau method is also equivalent to a Galerkin method, since Ey, is 
orthogonal with respect to (1 — x?)~!/? to all polynomials of degree up to q, 
as a consequence of (11.32). Note that the Galerkin method proper is more 
robust than equivalent tau or Chebyshev series methods, since, for example, 
it is unnecessary to introduce 7 terms or to find and use the form (11.32). 
The Galerkin method directly sets up a linear system of equations for its 
coefficients. For example, if we wish to solve 


u—-u=0, u(0)=1 (11.35) 
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by a Galerkin procedure using Legendre polynomials P(x) appropriate to 
(0, 1], namely 


un tn = >_eP*(z), (11.36) 
i=0 
then we solve 
1 
i: (ty, — Un). P*(a)da=0 (¢=0,1,...,n-1) (11.37) 
0 


and 


5 c, P*(0) = 1. (11.38) 
1=0 


Here (11.37) and (11.38) comprise n + 1 equations for co,...,Cn. Note that 
a snag in the Galerkin method is the need to evaluate the various integrals 
that occur, which are likely to require a numerical treatment except in simple 
problems such as (11.35). 


It is worth remembering that Chebyshev series are also transformed Fourier 
series, and so Chebyshev methods may be based on known methods for gen- 
erating Fourier partial sums or Fourier transforms, based on integrals and 
expansions. 


11.6 Collocation/interpolation and related methods 


We have seen, in Sections 5.5 and 6.5, that a Chebyshev series partial sum of 
degree n of a continuous function is a near-minimax approximation on [—1, 1] 
within a relative distance of order 47~? logn, whereas the polynomial of de- 
gree n interpolating (collocating) the function at the n+ 1 zeros of Tn41(x) 
is near-minimax within a slightly larger relative distance of order 27~! log n. 
Thus, we may anticipate an error that is 7/2 times as large in Chebyshev in- 
terpolation compared with Chebyshev series expansion. In practice, however, 
this is a very small potential factor, and polynomial approximations from 
the two approaches are virtually indistinguishable. Indeed, since collocation 
methods are simpler, more flexible and much more generally applicable, they 
offer a powerful substitute for the somewhat more mathematically orthodox 
but restrictive series methods. 


The title pseudo-spectral method was introduced by Gottlieb & Orszag 
(1977), in place of Chebyshev collocation method, to put across the role of this 
method as a robust substitute for the spectral method. Both series (spectral) 
and collocation (pseudo-spectral) methods were rigorously described by Ma- 
son (1970) as near-minimax. Note that minimax procedures generally involve 
infinite procedures and are not practicably feasible, while spectral, and more 
particularly pseudo-spectral, methods are typically linear and very close to 
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minimax and therefore provide an excellent and relatively very inexpensive 
substitute for a minimax approximation method. 


It has long been realised that collocation for differential equations is almost 
identical to series expansion. Lanczos (1957) noted that the ODE error form 
adopted in his tau method (11.32) could conveniently be replaced with nearly 
identical results (though different 7 coefficients) by 


Eryn = Ty41(x).(71 + Tet + +++ + 7.2871), (11.39) 


where q+ is the degree of Fy,. Note that the error in the ODE vanishes at 
the zeros of T,+41(x), and so the method is equivalent to a collocation method 
(in the ODE). Lanczos called this method the selected points method, where 
the zeros of Ty, are the points selected in this case. Lanczos sometimes also 
selected Legendre polynomial zeros instead, since in practice they sometimes 
give superior results. 


We have already shown that the Chebyshev collocation polynomial, f,,(«) 
of degree n to a given f(a), may be very efficiently computed by adopting a 
discrete orthogonalisation procedure 


f(z) © fn(z) = S~ aTi(2), (11.40) 
i=0 
where 
pes eee 
a=7 S > f(a) Ti =F Sf f (cos(@,)) cos(i6,), (11.41) 
k=0 k=0 
with ae 
Lk = cos(O;,) = cos (ar) (k =0,1,...,n). (11.42) 


For N = n, this yields the collocation polynomial, and this clearly mimics 
the Chebyshev series partial sum of order n, which has the form (11.40) with 
(11.41) replaced by 


— =fa — £7)? f(2)T; (x) da = =f f(cos(@)) cos(#0) dO. (11.43) 


with x = cos(6). 


Note that the discrete Chebyshev transform in x and the discrete Fourier 
transform in 0, that appear in (11.41), also represent an excellent numerical 
method (Filon’s rule for periodic integrands) for approximately computing 
the continuous Chebyshev transform and Fourier transform that appear in 
(11.43). The fast Fourier transform (FFT), which is of course a very efficient 
method of computing the Fourier transform, does in fact compute the discrete 
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Fourier transform instead. However, (11.43) is typically replaced by (11.41) 
for a value of N very much larger than n, say N = 1024 for n = 10. So 
there are really two different discrete Fourier transforms, one for N = n 
(collocation) and one for N >> n (approximate series expansion). 


11.7 PDE methods 


We note that, for simplicity, the above discussions of nomenclature have been 
based on ODEs, for which boundary conditions apply at just a few points, 
usually only one or two. Typically these boundary conditions are imposed 
exactly as additional constraints on the approximation, with only a little effect 
on the number of coefficients remaining. For example, in the tau method for 


Eu=u'-—u=0, u(0)=1- in (0,1, (11.44) 
we determine 


U~ Un = CofE ++++ + Cpa” 


by equating coefficients of 1,2,27,...,2” in 


Eun = (c1 — co) + (2c2 — c1)a@ + (3c3 — c2)a* Fees + 
+ (ney — Cn—1)2"—* — cna” = TT* (2). (11.45) 


This yields n+ 1 linear equations for co,...,¢n,7, and an additional equation 
is obtained by setting co = 1 to satisfy the boundary (initial) condition. 


In spectral and pseudo-spectral methods for PDEs, the boundary condi- 
tions play a much more crucial role than for ODEs, and it becomes important 
to decide whether to satisfy the boundary conditions implicitly, in the form 
chosen for the basis functions, or to apply the boundary conditions as addi- 
tional constraints. For this reason, Gottlieb & Orszag (1977) and Canuto et al. 
(1988) differentiate between Galerkin and tau methods primarily in terms of 
their treatment of boundary conditions —- whereas we have above viewed 
these methods as equivalent, one based on the orthogonality requirement and 
the other based on the form of the ODE (perturbation) error. Canuto et al. 
(1988) view a Galerkin method as a series method in which the boundary 
conditions are included implicitly in the chosen solution form, whereas a tau 
method is seen by them as a series method for which the boundary conditions 
are applied explicitly through additional constraints. 


The distinction made by Canuto et al. (1988) between Galerkin and tau 
methods has virtues. In particular the number of free approximation coeffi- 
cients needed to satisfy boundary conditions can be very large, whereas this 
may be a peripheral effect if the boundary can be treated implicitly. So a 
distinction is needed. But the words, Galerkin and tau, do not conjure up 
boundary issues, but rather an orthogonality technique and tau perturbation 
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terms. A better terminology, we would suggest, would be to refer to methods 
which include/exclude boundary conditions from the approximation form as 
implicit /explicit methods respectively. We could alternatively use the titles 
interior /mixed methods, as discussed for the MWR above. 


Nomenclature and methods become more complicated for PDEs in higher 
dimensions. In the following sections we therefore give a number of exam- 
ples of problems and methods to illustrate the formalisms that result from 
approaches of the Galerkin, tau, collocation, implicit, explicit (&c.) variety. 
We do not view spectral and pseudo-spectral methods, unlike Galerkin and 
tau methods, as specifically definable methods, but rather as generic titles for 
the two main branches of methods (series and collocation). A generalisation 
of the Lanczos tau method might thus be termed a spectral explicit /mixed 
tau method. 


11.7.1 Error analysis 


Canuto et al. (1988) and Mercier (1989), among others, give careful attention 
to error bounds and convergence results. In particular, Canuto et al. (1988) 
address standard problems such as the Poisson problem, as well as individu- 
ally addressing a variety of harder problems. In practice, however, the main 
advantage of a spectral method lies in the rapid convergence of the Chebyshev 
series; this in many cases makes feasible an error estimate based on the sizes 
of Chebyshev coefficients, especially where convergence is exponential. 


11.8 Some PDE problems and various methods 


It is simplest to understand, develop and describe spectral and pseudo-spectral 
methods by working through a selection of problems of progressively increas- 
ing complexity. This corresponds quite closely to the historical order of devel- 
opment, which starts, from a numerical analysis perspective, with the novel 
contributions of the 1960s and is followed by the fast (FFT-based) spectral 
methods of the 1970s. Early work of the 1960s did establish fundamental 
techniques and compute novel approximations to parabolic and elliptic PDEs, 
based especially on the respective forms 


u(x,t) ~ Un(x,t) = Sr eifité)E(2) (-l<a<1;t>0) (11.46) 
i=0 


for parabolic equations, such as uzz = uz, and 
ulry) ~unloy) = Vey H@)Ty) (1s2yst) (aay) 
i=0 j=0 


for elliptic problems, such as Au = ger + Uyy = f. 
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An early paper based on (11.46) was that of Elliott (1961), who deter- 
mined f;(£) as approximate solutions of a system of ODEs, in the spirit of the 
“method of lines”. Another early paper based on (11.47) was that of Mason 
(1967), which solves the membrane eigenvalue problem (see Section 11.8.2 
below) 


Au+Au=0in S, w=0on0S8, (11.48) 


for the classical problem of an L-shaped membrane (consisting of three squares 
co-joined), based on a preliminary conformal mapping of the domain and an 
approximation 


ud A(x, y)-bn(2, y), (11.49) 


where A = 0 is the algebraic equation of the mapped boundary. Mason 
(1969) also used an approximation of form (11.46) to solve a range of separable 
PDEs including (11.48). Indeed the leading eigenvalue of the L-membrane was 
computed to 13 significant figures by Mason (1969) (A = 9.639723844022). 


These early quoted papers are all essentially based on the collocation 
method for computing coefficients c; or cj. It is also possible to develop 
tau/series methods for the form (11.46), based on the solution by the Lanczos 
tau method of the corresponding ODEs for f;(t) ; this has been carried out for 
very basic equations such as the heat equation and Poisson equation (Berzins 
& Dew 1987). 


11.8.1 Power basis: collocation for Poisson problem 


Consider the Poisson problem 


o* 0? 

Aus at 5p = fey) in S, uw=0o0ndS, (11.50) 
where S is the square with boundaries « = +1, y= +1. Suppose we approxi- 
mate as 

m—2n—-2 
UL Umn = O(2,y) S- xe ayx'y’ , (11.51) 


i=0 j=0 


where we adopt the power basis xy? and include a multiplicative factor (x) 
such that ¢ = 0 is the (combined) equation of the boundaries. In this case, 


o(a,y) = (2? —1)(y?- 2). (11.52) 


Then we may rewrite Umn as 


m—2n—-2 


tonn = Yd aig (2? — a! \(y? — y’) (11.53) 


i=0 j=0 


274 CHAPTER 11: SOLUTION OF PARTIAL DIFFERENTIAL EQUATIONS 


and hence, applying A, obtain 


m—-2n—-2 
Aun = SY. Yoaig (+26 + a? — i — Yje*2y4(y? = 1) 
i=0 j=0 
PGE he GS Dee ae). (C64) 
Now set Atmn equal to f(x,y) at the (m— 1)(n — 1) points (x,y) (kK = 
1,...,m—1;1=1,...,n—1), where {z,}, {y} are the respective sets of zeros 


of Tn—1(x), Tr—1(y), respectively, namely the points 


ry = C08 a m= cos (S—) (11.55) 


This leads to a full linear algebraic system of (m—1)(n—1) equations for a,;. 
It is relatively straightforward to code a computer procedure for the above 
algorithm. 


We also observe that the approximation u,,, adopted in (11.53) above 
could equally well be replaced by the equivalent form 


timn = > > aij(a* — a" mod apf <q BON?) (11.56) 


i=2 j=2 


where (i mod 2) is 0 or 1 according as i is even or odd, since x? — 1 and y? —1 
are in every case factors of Umn. This leads to a simplification in Aumn (as 
n (11.54)), namely 


Atimn = ae (i—l)x Fe (a gl Oe) 


t=2 jJ=2 
+ j(j — Dy? 7 (at — at mot?) (11.57) 


The method then proceeds as before. However, we note that (11.51) is a more 
robust form for more general boundary shapes 0S and more general boundary 
conditions Bu = 0, since simplifications like (11.56) are not generally feasible. 


The above methods, although rather simple, are not very efficient, since no 
account has been taken of special properties of Chebyshev polynomials, such 
as discrete orthogonality. Moreover, (11.53) and (11.56) use the basis of power 
functions xy! which, for m and n sufficiently large, can lead to significant loss 
of accuracy in the coefficients a;;, due to rounding error and poor conditioning 
in the resulting linear algebraic system. We therefore plan to follow up this 
discussion in a later Section by considering more efficient and well conditioned 
procedures based on the direct use of a Chebyshev polynomial product as a 
basis, namely T;(x)T;(y). 

However, before we return to the Poisson problem, let us consider a more 
difficult problem, where the form (11.51) is very effective and where a power 
basis is adequate for achieving relatively high accuracy. 
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11.8.2 Power basis: interior collocation for the L-membrane 


Consider the eigenvalue problem 
Au+Au=0inS, u=O0ondS, (11.58) 


where S is the L-shaped region shown (upside down for convenience) in Fig- 
ure 11.2. It comprises three squares of unit sides placed together. To remove 
the re-entrant corner at O, we perform the mapping, adopted by Reid & Walsh 
(1965), 

Zaz (2h =a'+iy, z=ar+iy), (11.59) 
where x,y are coordinates in the original domain S (Figure 11.2) and 2’,y’ 
are corresponding coordinates in the mapped domain $” (Figure 11.3). 


C B 


ra 
O A 
D E 
Figure 11.2: L-membrane Figure 11.3: Mapped domain 


Note that the domain S’ is the upper half of a curved hexagon with corners 
of angle $ shown in Figure 11.3, where the vertices A’, B’,C’, D’, E" corre- 
spond to A,B,C, D,E in Figure 11.2. (The lower half of the hexagon does 
not concern us, but is included in Figure 11.3 to show the geometry and the 


symmetry.) Now, from the mapping, 


Pagel. OC 20. (11.60) 
Then 
Autdu = eae uals r | 
= tlre (Se) toe] 
= (r)-3 E ("2) : (“S*) is sa | + Xu. (11.61) 
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Hence 
Au + du= $(r')7* (A'u + Sr’du) = 0, (11.62) 


where dashes on A’,r’ indicate that dashed coordinates are involved. Thus 
the problem (11.58) has transformed, now dropping dashes on r,u, to 


Aut 4rd\u=0in S’, w=00n 0S". (11.63) 


Before proposing a numerical method, we need to find the algebraic equa- 
tion of the boundary O’ A’B’C’D'E"(O’) in Figure 11.3. This boundary has 
two parts: the straight line E’O’A’, namely y’ = 0, and the set of four curves 
A'B'C'D' E’ which correspond to (~?—1)(y?—1) = 0 in S. Thus the boundary 
equation is 

0 = Alz,y) =4y'(2?-Dy’?-1)= ears 
y’ (r+ sin?(20) — 4r? + 4) = y! [( [(r’)® sin? (36’) — 4(r’)? + 4] 


y' |v’ P8(e'? - WP P = 4{@'? ee (11.64) 


Dropping dashes again, 


A(x, y) = y. [y?(3x? y2)? — A(x? + y?)3/? 4 4] =0. (11.65) 


We now adopt as our approximation to u, using (11.65) for ¢: 


m n 


UX Umn = A(z, y). x Se catty) , (11.66) 


i=0 j=0 
where t = 0 or 1, according as we seek a solution which is symmetric or 
anti-symmetric about OC. For the leading (i.e., largest) 4, we choose t = 0. 
The rectangle TUVW, which encloses the mapped membrane, has sides 
O'T, TU in the zx, y directions, respectively, of lengths a,b, say, given by 
a = O'T =O'B' cos(r/6) = 21/931/2 /2 = 2-7/331/2, 
b TU =O'C! = (21/?)?/8 = 21/8, (11.67) 


An appropriate ‘interior’ collocation method is now simply constructed. We 
specify that the form of approximation (11.66) should satisfy the PDE (11.63) 
at the tensor product of (m+ 1)(n+1) positive zeros of T* . ,(x/a)T7,,(y/b), 
where a,b are given in (11.67), namely the points 


(k=1,...,m41; l=1,...,n+)). (11.68) 
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Table 11.1: Estimates of first 3 eigenvalues of L-membrane 


m=n Rayleigh quotient 


Functional form A(z, y) a pa ariy 


15.2159 

15.1978 15.1980 
15.1974 15.1974 
15.1974 15.1974 


19.8054 
19.7394 
19.7392 19.7392 
19.7392 19.7392 


This leads to a homogeneous system of (m-+1)(n+ 1) linear equations, which 
we may write in matrix form as A.c = 0, for the determination of c = {c,;}, 
where A depends on ». The determinant of A must vanish, thus defining 
eligible values of A, corresponding to eigenvalues of the PDE. We have applied 
the secant method to find the \ nearest to a chosen guess. Results for the first 
three eigenvalues, taken from Mason (1965), are shown in Table 11.1 together 
with Rayleigh quotient estimates. Clearly the method is rather successful, 
and the application serves as an interesting model problem. 


Strictly speaking, the collocation method above is not an interior method, 
since some collocation points are exterior to S although interior to the rect- 
angle TUVW. However, the PDE solution does extend continuously across 
the problem boundaries to these exterior points. 


In fairness we should point out that, although the above collocation method 
is probably at least as effective for this problem as the best finite difference 
method, such as that of Reid & Walsh (1965), it is not the best method of 
all. A better method for the present problem is the boundary method, based 
on separation of variables, due to Fox et al. (1967) and further extended by 
Mason (1969). This breaks down for regions with more than one re-entrant 
corner, however, on account of ill-conditioning; a better method is the finite- 
element /domain-decomposition method described by Driscoll (1997). 
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11.8.3. Chebyshev basis and discrete orthogonalisation 


In the remaining discussion, we concentrate on the use of a Chebyshev poly- 
nomial basis for approximation and exploit properties such as discrete orthog- 
onality and the FFT for efficiency. However, it is first appropriate to remind 
the reader that the classical method of separation of variables provides both 
a fast boundary method for Laplace’s equation and a superposition method, 
combining interior and boundary methods for the Poisson equation with non- 
zero boundary conditions. 


Separation of variables: basic Dirichlet problem 


Consider the basic Dirichlet problem for Laplace’s equation on a square, 
namely 


Au=0in S, (11.69a) 
u = g(x,y) on OS, (11.69b) 
where OS is the square boundary formed by « = +1, y = +1, S is its interior 


and g is defined only on 0S. Then we may solve (11.69a) analytically for the 
partial boundary conditions 


u=g(-l,y)onx=-1; w=Oonev=41,y=-—l,y=41, (11.70) 


in the form 


u= Sax sinh $kn(1— x) sin $kx(1—y), (11.71) 
k=1 
where a, are chosen to match the Fourier sine series of g(—1,y) on x = —1. 
Specifically 
g(-1,y) = So be sin $kr(1 — y), (11.72) 
k=1 
where : 
by = 2 | g(—-1,y) sin $kr(1 — y) dy, (11.73) 
Si 


and hence a,x is given by 


ay = by [sinh kr] 7+. (11.74) 


Clearly we can define three more solutions of (11.69a), analogous to (11.71), 
each of which matches g(x, y) on one side of 0S' and is zero on the remainder 
of 0S. If we sum these four solutions then we obtain the analytical solution of 
(11.69a) and (11.69b). For an efficient numerical solution, the Fourier series 
should be truncated and evaluated by using the FFT [see Section 4.7]. 
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Chebyshev basis: Poisson problem 


The Poisson Problem can be posed in a slightly more general way than in 
Section 11.8.3, while still permitting efficient treatment. In particular we may 
introduce two general functions, f as the right-hand side of the PDE, and g 
as the boundary function, as follows. 


Au= f(x,y) in S, w= g(x,y) in 0S, (11.75) 


where S and 0S denote the usual square {-1 < « < 1,-1 < y < 1} and its 
boundary. Then we may eliminate g (and replace it by zero) in (11.75), by 
superposing two problem solutions 


U=U,+ Ua, (11.76) 


where uz is the solution of the Laplace problem ((11.75) with f = 0) and ug 
is the solution of the simple Poisson problem ((11.75) with g = 0), so that 

Au, =0in S, uy = g(x,y) on OS, (11.77a) 

Aug = f(z,y) in S, ue =00n OS. (11.77b) 

We gave details above of the determination of u; from four Fourier sine series 

expansions based on separation of variables, as per (11.71) above. We may 


therefore restrict attention to the problem (11.77b) defining uz, which we now 
rename u. 


We now re-address (11.77b), which was discussed in Section 11.8.1 using 
a power basis, and, for greater efficiency and stability, we adopt a boundary 
method based on a Chebyshev polynomial representation 


se Mae: 


Umn = (@? — I(y?2 —1) SYS ey ), (11.78) 
j=0 


i=0 
or equivalently, again to ensure that u = 0 on 0S 


tint =) tO) —Tiasat| LG) —DeacaeG). G79) 
4=2 J=2 
Now, as in Problem 16 of Chapter 2, 
92 i-2 i. 
Fan = Ly -ni+ nn, (11.80) 
and hence 
mn 4-2 
Atimn = >>> ay | >> G-r)i@+r)T-(2) (Fj) — Tj moa 2(y)) + 
i=2 j=2 r=0 


i—r even 
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a x (J — 8)9(9 + 8)Ts(y) Til) — Ti moa 2(2)) 


=f (11.81) 


Now define collocation points {x,(k = 1,...,m—1)} and {y(J = 1,...,n- 
1)} to be, respectively, the zeros of Tm—i(a) and T,-1(y). Then discrete 
orthogonality gives, for p,r less than m — 1 and q,s less than n — 1, 


m—1 2, p=rTr= 0, 
2(m +1)? Tp(@ Se a, pHa G, (11.82a) 
k=1 0, pr, 
n-1 2, qs = 0, 
2(n+1)* YS Ty(m)Te(H) = 4 1, g=s 40, (11.82b) 
1=1 0, q#s. 


Evaluating (11.81) at (a,, yc), multiplying by 4[(m — 1)(n — 1)]~+ and by 
Tp(z)Tq(ye) for p = 0,...,m —2;q =0,...,n — 2, summing over k, é, and 
using discrete orthogonality, we obtain 


m—-l1n-1 
Apg + Bog = 4[(m (n—1))* S- Ss? fl f (te, ye)Tp(te)Tg(ye), (11.83) 
k=1 @=1 
where 
y Aig (t pia +p), q 2 2, 
i—p even 
A.=d- D4 ZC ayjli-—piiG+p), @=1, 
Pq =2 
i= j 


De ap (9 )i(G +4) p22, 
=) 
Pe even 
PY  j=2 a 


25 FF ayG-gii +a, p=0. 


(11.84) 
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This system of linear equations for a;; is very sparse, having between 2 and 
(m+n — 2)/2 entries in each row of the matrix for p,q > 2. It is only the 
equations where “boundary effects” enter (for p = 0,1; q = 0,1), that fill 
out the matrix entries into alternate rows and/or columns. Note also that all 
right-hand sides are discrete Chebyshev transforms, which could be evaluated 
by adopting FFT techniques. 


The border effects can be neatly avoided for this particular Poisson prob- 
lem, by adopting instead a matrix method based on differentiation matrices, in 
which the unknowns of the problem become the solution values at Chebyshev 
nodes, rather than the solution coefficients. This approach was introduced 
in Section 10.5.3 of Chapter 10 for ODEs and is particularly convenient for 
some relatively simple problems. We now see its advantages for the present 
problem. 


11.8.4 Differentiation matrix approach: Poisson problem 


To illustrate this approach, we follow the treatment of Trefethen (2000), set- 
ting m = n and adopting as collocation points the tensor product of the zeros 
of (1 — 2?)U;-1(x) and the zeros of (1 — y?)Un—1(y). The reader is referred 
to Section 10.5.2 for a detailed definition of the (n+ 1) x (n+ 1) differentia- 
tion matrix D = Dy, which transforms all u values at collocation points into 
approximate derivative values at the same points, by forming linear combina- 
tions of the u values. The problem is significantly simplified by noting that 
the border elements of D, namely the first and last rows and columns of Dy, 
correspond to zero boundary values and may therefore be deleted to give an 
active (n — 1) x (n—1) matrix Dy. 


For the Poisson problem 


Au = f(z,y) nS: {-1l<a2<1,-l<y<l}, (11.85a) 


u=O0ond0S: {x=+l,y=+}]}, (11.85b) 


the method determines a vector u of approximate solutions at the interior 
collocation points (compare (10.52) with eg = e, = 0) by solving 


E,u= f, (11.86) 


where 7 7 
E, =1@D?2+D? eI (11.87) 


and A @ B is the Kronecker (tensor) product, illustrated by the example 


aa alba 6b 
a b a{ B\ {| ay aod | by bé 
c d y Of 
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For example, for n = 3, this gives (see (10.49) et seq.) 


E; = 1@D2 + D2@ 1 


-14 6 -2 
i Gs a 
= 0: 26: -a aad 
214 6) 22 
= Ay 2G: A of 
SG 14 
HAG 
4 -6 4 
-2 6 -14 
214 6 =) 
=14 6 75 
iA 6 =) 
4 —6 4 
+ 4 -6 4 . (11.88) 
4 -6 4 
=) 6 =A 
29 6 -14 
=) 6 =[4 


Clearly the matrix E,, is sparse and easily calculable. Hence (11.86) may 
readily be solved by Gauss elimination, using for example Matlab’s efficient 
system. The right-hand side of (11.86) is simply obtained by evaluating f at 
the interior collocation points. The shape (location of the non-zero terms) of 


Figure 11.4: Shape of matrix E,, for n = 3 


x RM XxX] xX x 
x XK XxX x x 
x XK XxX x 
x x 
x x 
x x 
x x % XxX 
x x x %% xX 
x x |X x*% x 


the matrix E,, is illustrated in Figure 11.4 and consists of a diagonal of square 
matrices, flanked by diagonal matrices. The matrix is very sparse in all rows, 
having 2n — 3 non-zero entries out of (n — 1)?. 
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This Differentiation Matrix method is very attractive and efficient for this 
problem, and should always be given consideration in problems that respond 
to it. We now turn our attention to a more general problem, with non-zero 
boundary conditions. 


11.8.5 Explicit collocation for the quasilinear Dirichlet problem: 
Chebyshev basis 


We now continue to exploit the better conditioning of a Chebyshev polynomial 
basis, but we also consider the greater generality of a Dirichlet problem for a 
quasilinear elliptic equation on a square, namely 


Lu = G.Uge + b.tgy + 6.Uyy + dg +e-uy = fin D: |x| <1,|y| <1, (11.89a) 


u= g(x,y) on OD: {x = +1,y = +1}, (11.89b) 


where a, b,c,d,e, f are functions of x and y defined in D, g(x,y) is defined on 
OD only, and where, to ensure ellipticity, 


a.c > b? for all (x,y) in D. (11.90) 


This is an extension of recent work by Mason & Crampton (2002). 


For generality we do not attempt to include the boundary conditions 
(11.89b) implicitly in the form of approximation, but rather we represent 
them by a set of constraints at a discrete set of selected points, namely Cheb- 
yshev zeros on the boundary. Moreover we adopt a Chebyshev polynomial 
basis for u: 


Gergen a S~ aij Ti(a)T;(y). (11.91) 


As it happens, we find that the apparently most logical collocation pro- 
cedure, similar to that of Section 11.8.1 above, for approximately solving 
(11.89a), (11.89b) in the form (11.91), leads to a singular matrix and re- 
quires modification. More details about this follow as the method develops. 
The fundamental idea that we use is that, since umn, given by (11.91), has 
(m +1)(n +1) undetermined coefficients, we expect to be able to generate 
an appropriate set of equations for a;; if we form (m — 1)(n — 1) equations 
by collocating (11.89a) at a tensor product of Chebyshev zeros and a further 
2m + 2n equations by collocating (11.89b) at Chebyshev zeros on the bound- 
ary. It is in the formation of the latter boundary equations that difficulties 
arise, and so we consider these equations first, noting that they are completely 
independent of the specification Lu = f of the elliptic equation (11.89a). 


To form the 2m + 2n boundary equations for a;;, we set Umn equal to g 
at the zeros, X, (k = 1,...,m) and Ye (€ = 1,...,n) of T(x) and T,(y), 
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respectively, on y = +1 and x = +1, respectively. This gives the two pairs of 
equations 


/ 


aij Ti(Xp)T; (+1) 7 Xr; eel); 


Jj 


S~ ai TH(EUT;(Ye) = g(£1, Yo). (11.92) 
i=0 jj 


n 


0 
n 
=0 


If we add/subtract these pairs of equations, noting that Tj(1) = 1 and that 
T;(—1) = (—1)/, we deduce that 


n 
is / 


S~ aij Ti(Xe) = Guo = 4(G(Xe, 1) + 9(Xe, -1)), (k= 1,...,m) 
S~ aijTi(Xt) = Gar = 2(G(Xu, 1) — (Xe, —1)), (k= 1,---,m) 


=1 

odd 

S> aijTj(Ve) = Hoo = $(9(1, Ye) + g(-1, ¥2)), (€ =1,...,n) 
j=0 


(11.93) 


where the arrays Gy», Heg are defined above for p = 0,1; ¢ = 0,1. 


Now, defining w; to be 2/(¢+ 1) for all 2, multiplying the first pair of equa- 
tions in (11.93) by wmT,-(X,) and summing over k, multiplying the second 
pair of equations by w,T,(Y¢) and summing over @, and exploiting discrete 
orthogonality, it follows that 


n ; m 
Rro = oe ari = Ing = Wm > Tr(Xt)Geo, (r = 0,...,m—1) 
j=0 k=1 
j even 
Rr = S- ari = Jrt = Wm > Tr(X4)Ger, (r = 0,...,m—1) 
j=1 k=], 
j odd 


uae 
Cso = S- Gis = Koo = Wn )_Ts(Ye) Heo, (s =0,...,n— 1) 
i=0 


4 even 
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> ais = Kei = wn STs (Vo)Ha1, (s =0,...,n—1) (11.94) 

i=1 é=1 

4 odd 
where R, C, J, K are defined to form left-hand sides (R, C) or right-hand sides 
(J, K) of the relevant equations. In addition each R or C is a linear sum of 
alternate elements in a row or column, respectively, of the matrix A = [aj,]. 


Now we claim that this set of 2m+2n linear equations (11.94) in ago, .--, @mn 
is not of rank 2m + 2n but rather of rank 2m + 2n— 1. Indeed, it is easy to 
verify that a sum of alternate rows of A equals a sum of alternate columns; 
specifically 


> ; 


ss Rip = 3 Ciq = pS = Aig, (11.95) 
ay Baa m— toda m— Fade Lr 


where p = 0,1 for m— 1 even, odd, respectively, and g = 0,1 for n — 1 even, 
odd, respectively. For example, for m =n = 4, 


Ry + R31 = Cy + C31 = ay, + 213 + 231 + 233, (11.96) 

and, form =n = 3, 
$Roo + Roo = $Coo + Cro = $(Fa00 + 02 + A290) + a2. (11.97) 
Clearly we must delete one equation from the set (11.94) and add an 
additional independent equation in order to restore full rank. For simplicity 


we shall only discuss the cases where m,n are both even or both odd, leaving 
the even/odd and odd/even cases to the reader. 


For m,n both even, we delete Ci, = Ky, from (11.94) and add an averaged 
“even/even” boundary collocation equation 


+[u(1, 1) + u(—1, 1) + u(-1, -1) + u(1, -1)] = 
doo = F[g(1, 1) + g(-1,1) + g(—-1, -1) + g(1, -1)]. (11.98) 


This simplifies to 


yy Rio = Xoo (11.99) 


where Ryo is defined by extending the definition (11.94) of R,o to r = m and 
where Ago is as defined in (11.98). We may eliminate every R except Rmo 
from this equation, by using (11.94), to give a simplified form for the extra 
equation 


Rmo = Imo = Aoo — 3 Jio (11.100) 


a ae 
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where the right-hand side Jmo is defined as shown. 


For m,n both odd, we delete Coo = Koo from (11.94) and add an averaged 
“odd/odd” boundary collocation equation 


+[u(1, 1) u(—1,1)+u(-1 
Maa = H(t) — (1,14 9(-1,-1)-9(,-1)]. (11.101) 


This simplifies to 


Ss) Cn = Au (11.102) 
j=l 

j odd 
where C;,; is defined by extending the definition (11.94) of C,1 to s = n and 
where Aj; is as defined in (11.101). 


We may eliminate every C except C;,; from this equation, by using (11.94), 
to give a simplified form for the extra equation 


n—-2 
Chi = Kni = All — SS Ky (11.103) 
fad 
where the right-hand side K,, is defined as shown. 


We now have 2m + 2n non-singular equations for the coefficients a;;, and 
it remains to handle the elliptic equation by collocation at (m — 1)(n — 1) 
suitable Chebyshev points in D. 


For a general quasilinear equation we should set Lu = f at a tensor product 
of (m—1) x (n—1) Chebyshev zeros, giving the same number of linear algebraic 
equations for {a;;}, and these equations together with the 2m + 2n boundary 
collocation equations would then be solved as a full system. 


For simplicity, and so that we can give fuller illustrative details, we concen- 
trate on the Poisson equation, as a special example of (11.89a), corresponding 
to the form 

Lu = Au = Ure + Uyy = f(x,y) in D. (11.104) 


Now second derivatives of Chebyshev sums are readily seen (see Chapter 2) 
to take the form 


2 k-2 
Tila) = Sv (k—nk(k+r)T(x) (k> 2), (11.105a) 
r=0 
k—r even 
a2 es 
ap) = » (€—s)@(l+s)T.(y) (€> 2). (11.105b) 
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Hence, on substituting (11.91) into (11.104), we obtain 


m n k—-2 
/ / 
Atmn = S- S- areTe(y) > (k —r)k(k + r)T,(a) 

k=2 £=0 r=0 

k—r even 

n m , ae 

+S 2 So aneTe(x) S > (€— sel + s)Ts(y) 
f=2 k=0 s=0 
é—s even 
= Amn, say. (11.106) 


Setting Aumn equal to f(x,y) at the abscissae x;,y; , where x; are zeros of 
Tm—1(£) and y; are zeros of T,-i(y) (fori =1,...,m—1; 7 =1,...,n—1), 
multiplying by T,(x;)T,(y;), and summing over i, 7, we deduce that, for every 
p=0,...,m—2; q=0,...,n—2: 


m—1ln-1 
Eng — Arn (2i543) Tole) Ta(yd) 
i=1 j=l 
m—-ln-1 
= f(x, Yj) Tp (vi) Ta (ys) 
i=1 j=1 
= foo (11.107) 


where fpg represents the discrete Chebyshev transform of f with respect to 
T,(x)Tq(y). Substituting for Amn, 


m—-1ln-1 m n k-2 
/ / 
Enq = S75 axeTe(yj)Talys) S> (k= r)k(k +r) T (ai) Tp (xi) 
i=1 j=1 k=2 C=O r=0 
m—-iln-1 n ar fae 
# SSS aneTe(ws)Tp(as) 55 (€— s)€(E + 8)T(yj)Ta (5s) 
i=1 j=l €=2 k=0 s=0 
m non-l k-2 m—1 
’ ! 
=> To(yy)Ta(ysane D> (k—r)k(k +7) $7 T,(xi)Tp(aa) 
k=2 @=0 j=l r=0 i=1 
k—r even 
n m ,mai abe n-1 
+S So SS Tilws)Tp(wilane SS (0 - 8)e(C +8) $- Ts(y;)Ta(ys)- 
(=2 k=0 i=l s=0 qi 
f—s even 
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Using the discrete orthogonality property that, for example, 


n=1 0, LAq 
SS" Te(ys)Ta(yj) = 4 (n-1)/2, C= #40 , 
j=l n—1, l=q=0 
we deduce that 
m n-1 
Dee S- 4(n— 1)dzg + Ss Tn—1(Yj) Tq (Yj )@kn-1 
k=p+2 j=l 
k—p even 
n-1 
+ S°Tn(yj)Ta(¥y)axn | $(m —1)(k — p)k(k + p) 
j=l 
n m—-1 
+} So $(m=lape t+ So Tn—1(ai)Tp(#i)am—10 
l=q4+2 i=1 
—q even 
m—-1 
+ S Tm (Xi) Tp (Xi) ame 3(n —1)(¢-— gee+aq) 
i=l 


(11.109) 


Now, by the definition of x;,y;, we know that T,,-1(x,;) and T,_1(y;) are 
zero. Also, using the three-term recurrence at x;, 


Tm (@i) = 2@Tm—1(&) — Tm—2(i) = —Tm-2(#i), Tn(yj) = —Tn-2(ys)- 
(11.110) 
Substituting these values into (11.109), using discrete orthogonality, and using 
the Kronecker delta notation 


Ors = 1 (r=8), Ors =O (r ¥ 8), (11.111) 
we deduce that 
Epq = ¢(m—1)(n—-1)| SY) (eq — dgn—20en) (4 — p(k + p) + 
i oataee 
—p even 


a ye (pt — bp,m—24m1) (1 — gil + | 


= foq (p=0,...,m—2; ¢q=0,...,n—2). (11.112) 
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For example, for m = n = 3 we have this set of four collocation equations: 


Ey 


Eo 


= Flas — a33) + (a13 - a33)|2.3.4 


4 


= 24(a13 + a31 — 2033) = fir = 24F in, 

= a392.3.4 + (aig — a32)2.2.2 

= 8(a12 + 3a30 — 432) = fio = 8Fio, 

= (a1 — G23)2.2.2 + ao32.3.4 

= 8(3a03 + G21 — 423) = for = 8For, 

= A02.2.2 + ag22.2.2 

= 8(a02 + a20) = foo = 8Foo, (11.113) 


where F;; are defined as shown by scaling fi;. For m = n = 4, the sys- 
tem (11.112) gives the following nine equations, where we leave the reader to 


confirm the details: 


Eo 


Eg, 


Eo 


= 108(a24 + Gaz — 2044) 

= fro = 108F 2, 

= 54(ao3 + 2a41 — a43) 

= for = 54Fo, 

= 18(ag2 + 824 + 6a49 — G42 — 844) 
= foo = 18Fr, 

= 54(2a14 + a32 — 3a) 

= fio = 54F 12, 

= 54(a13 + a31) 

= fi = 54F i, 

= 18(a12 + 8a12 + 3a30) 

= fio = 18F io, 

= 18(6ao4 + a22 — ae4 + 8a42 — 8a44) 
= fo2 = 18Fo2, 

= 18(3a03 + aa1 + 8441) 

= for = 18Fo1, 

= 18(ao2 + 8a04 + @20 + 8a40) 


= foo = 18F pp. (11.114) 


For m = n = 4, the complete set of 25 collocation equations, 16 boundary 
equations and 9 interior PDE equations, namely (11.94) for m = n = 4 and 
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(11.114), may be written in the matrix form 
Ma =b, (11.115) 


where M is the matrix of equation entries and a is the column vector of 
approximation coefficients 


a= (doo, Q01,---,404, 410, 411,---, 414, 420,---,430,---,440,---,; aaa)’ 

(11.116) 
and b is the set of right-hand sides, either boundary or PDE terms, in appro- 
priate order. In Table 11.2 we display the matrices M, a, b form = n = 4, 
blank entries denoting zeros. The column of symbols to the left of M indi- 
cates which equation has been used to construct each row. Note that we have 
ordered the equations to give a structure in M as close to lower triangular as 
possible. The order used is based on: 


Ras, R3x, Ex, Rox ’ Ey ’ Rix, Lox; Rox; C3, Coax, Cis, Cox (11.117) 


where * is a wild subscript, F indicates a PDE term, and R, C indicate bound- 
ary conditions. 


On studying Table 11.2, some important facts emerge. The coefficients 
aij appearing in any equation are exclusively in one of the four symmetry 
classes: i, 7 both even, i, 7 both odd, i odd and j even, and 7 even and 7 odd. 
Thus the set of 25 equations can be separated into four wholly independent 
subsystems, respectively involving 4 subsets of a;;. These four subsystems are 
shown in Table 11.3 for m = n = 4, and they consist of 8,5,6,6 equations in 
9,4,6,6 coefficients a;;, respectively. 


This immediately confirms that we have a surplus equation in the odd/odd 
subsystem (for a11, @13, 431, @33) and one too few equations in the even/even 
subsystem. As proposed in earlier discussions, we therefore delete equation 
Ci, and replace it by equation R49, as indicated in Table 11.2. 


The extremely sparse nature of the matrix M is clear from Table 11.2, and 
moreover the submatrices formed from even and/or odd subsystems remain 
relatively sparse, as is illustrated in Tables 11.3 to 11.6. 


The odd/odd subsystem (for m = n = 4) (in Table 11.6) is remarkably easy 
to solve in the case g = 0 of zero boundary conditions, when J,, = Ky, = 0. 
The solution is then 


—ai1 a13 a31 — 433 4Fy1. (11.118) 


In Table 11.7, we also show the full algebraic system for the odd degrees 
m =n = 3, and in Tables 11.8 to 11.11 the equations are separated into their 
four even/odd subsystems. The equation Coo is noted and is to be deleted, 
while the equation C3; has been added. Equations are again ordered so as to 
optimise sparsity above the diagonal. The m = n = 3 subsystems are easily 
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Table 11.2: Full collocation matrix—Poisson problem: m =n = 4 


Rao #01. 0: | Jao 
ao0 

Rai 01010 Ts 
1 aol 

R30 37010 1 J30 
ao2 

E22 00001 001 0 -2 Foo 
ao: 

E>, 0001 0 020-10 os Fo 
a 

Eg 00108 60-10 -8 i Fog 
a10 

Rat 01010 Joi 
ji a1 

R20 37010 1 J20 
a12 

E12 00002 0010-1 Fis 
a 

Ey, 00010 01000 2 Fu 
a 

E10 00108 3000 0 ~ Fig 
a20 

Ru 01010 Tig 
1 a21 

Rio 370101 Ji0 

a22 > 

Epo |0 0006 0010-1 0 8 -8 Foo 
a2: 

Ep. |0 0030 0100 8 0 0 o Fo. 
a 

Eo |0 0108 1000 00 0 SS Foo 
a: 

Ro. 01010 iz Jo1 
1 a31 

Roo 70101 Joo 
a: 

C30 00040 0001 0 000 1 0 a K30 
433 

Csi 00010 0001 0 Ky 
a: 

C21 00100 0010 0 a Koy 
1 a4o 

C20 05 0010 0 1 0 Kao 
7 aa 

Cro 10 0100 1 0 0 K10 
aa2 

Cu 01000 0100 0 Ki 
aas 

Cor 10000 10000 Kai 
Qa44 

Cor [4 0000 1000 0 100 0 04]7- Koz 
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Table 11.3: m =n = 4, partial system odd/even in x/y 


R30 311 a10 J30 
Fiz |00201-1 a2 Fiz 
Fio |01830 0 Gia] Fro 
Rio | $11 aso | | Sto 
Co, |01001 0 a32 Kai 
Co1 10010 0 Las34 Koi 


E21 012-1] [ao Foi 
Rai 11 a03 Joi 
Eo. |0 3108 0 aa} | Fo 
Ro. | 11 assy | | Sox 
C30 |0 4010 1 aai K30 
Cio L$ 0101 04 Laas Kio 


Table 11.5: m =n = 4, partial system even/even in x/y 


Rao 3 11 aoo | Jao 
E22 0010 1 -2 ao2 Foo 
E20 018 6-1-8] | aoa F20 
R20 311 a20 J20 
Eo2. |0060110 8 -8 a22 | = | Fo2 
Eo |0181008 0 0 a24 Foo 
Roo | $11 aao Joo 
Co 105001001 0 aa2 K20 
Co L$ 001001 0 O | Laaa L Koo 


R31 11 J31 
ai1 

Ei, |0110 Fir 
a13 

Rii 11 ea Sir 
a31 

C31 ;O 101 Kai 


a 
Ci 1010 Kui 
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Table 11.7: Full collocation matrix—Poisson problem: 


spaces contain zero entries) 
Car | 0001 
Rai 
R20 
Fis 0001 
Fo 0010 
Ri 0101 
Rio 3010 
Ep. |0 003 
Eoo | 0 010 
Ror |0 101 
Roo | 4 010 
Co |0040 
Car 0010 
Ci 0100 
Cio | 0 § 0 
Coo | $ 0 0 
Co 1000 


i=) 


Nie 


00 0 1 
01 0 -—2 
30-1 0 
00 1 0 
010 0 
10 0 O 


293 


m =n = 8 (blank 


Joi 
J20 
Fis 
Fro 
Si 
Jio 


Joi 
Joo 


een 


solved to give explicit formulae in the case g = 0, as we now show. We leave it 
as an exercise to the reader to generate a set of tables for the casem =n =5. 


We may readily determine formulae for all coefficients a,; form = n = 3 by 
eliminating variables in Tables 11.8 to 11.11, and we leave this as an exercise 
to the reader (Problem 7). 


We deduce from Table 11.8 that, for g = 0, and hence J,, = Ky. = 0, the 


even/even coefficients are 


2a02 2020 —4a22 Foo. 


— 00 


From Table 11.9, for g = 0, the odd/odd coefficients are 


—a11 a13 31 


From Table 11.10, for g = 0, the even/odd coefficients are 
—ao1 = 403 = 2a21 = —2a23 = $F. 


From Table 11.11, for g = 0, the odd/even coefficients are 


1 
—10 = 2012 = A309 = —2a32 = ZF io. 


(11.119) 


(11.120) 


(11.121) 


(11.122) 
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Table 11.8: m = n = 3, partial system even/even in x/y 


R20 5 J20 
@o00 
ao2 

Roo | $ 1 = | Joo 
a20 

C20 0 $ 1 K20 
7 a22 

Coo [4.010 Koo 


Table 11.9: m = n = 3, partial system odd/odd in a/y 


C31 }010 1 ai K31 
Fy }011-2] Jas} | Pa 
Riu {il asi | Ju 
Ci, [101 0 a33 Ki 


Table 11.10: m = n = 3, partial system even/odd in a/y 


Rei 1 1 ao1 Jar 
Eo. | 031-1 aos | _ | For 
Roi | 1 1 a2 ~ Joi 
Cio [$01 0 a23 Kio 


Fio |} 013-1 a10 Frio 
Rio $ 1 a12 aay Jio 
Car |010 1 a30 Ka 


Co1 101 0 a3z2 Ko. 
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Thus for zero boundary conditions, the approximate solution u is given 
very simply for m = n = 3. Indeed we see, not surprisingly, (Problem 8) that 
Umn can be written exactly in the form 


Umn = (a? — 1)(y* —1)(a + ba + cy + day). (11.123) 


If Jxx and K,, are not both zero, then no simplification such as (11.123) 
occurs, but we still obtain four separate sparse subsystems to solve for the 
coefficients a;; for all m, n. 


An alternative but closely related approach to the special case of the Pois- 
son problem is given by Haidvogel & Zang (1979). 


11.9 Computational fluid dynamics 


One of the most important PDE problems in computational fluid dynamics 
is the Navier-Stokes equation 

Ov 

BET eS PEA V-v=0 (11.124) 
where v is the velocity vector, p is the pressure divided by the (constant) den- 
sity, v is the kinematic viscosity and A denotes the Laplacian. This problem 
is studied in detail in the lecture notes by Deville (1984), as well as in Canuto 
et al. (1988). Deville considers, as preparatory problems, the Helmholtz equa- 
tion, the Burgers equation and the Stokes problem. We shall here briefly 
discuss the Burgers equation. 


The Burgers equation is the nonlinear equation 


Ou Ou Oru 


which we shall take to have the boundary and initial conditions 


u(—1,t) =u(1,t) =0, u(x,0) = uo(2). (11.126) 


The general procedure for solution is to discretise (11.125) into a first-order 
system of ordinary differential equations in t, which is solved by a scheme that 
is explicit as regards the nonlinear part and implicit for the linear part. Using 
Chebyshev collocation at the n+ 1 points {y;}, the discretisation can be 
written (Canuto et al. 1988) as 


O 
Zn oS +UnDntn — Duy) =0, (11.127) 
where D,, is the appropriate Chebyshev collocation differentiation matrix, 
u, is a vector with elements u,(y;,t), Un is a diagonal matrix with the 
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elements of u, on its diagonal and Z, is a unit matrix with its first and 
last elements replaced by zeros. The boundary conditions are imposed by 
requiring Un(yo,t) = Un(Yn,t) = 0. The method as it stands involves O(n?) 
operations at each time step for the implicit term, but this can be reduced to 
O(nlogn) operations by using FFT techniques. 


A Chebyshev tau method may instead be applied, defining 


_ 2 fh £@T2) g., 
(f, Tk) = = a Traut ot (11.128) 


Then, defining 


k=0 
we have A 3 Py 
( A ge a Th) =0, (11.129) 
which reduces to 
d Oun 


Again a mixed explicit /implicit method may be adopted for each time step, 
the inner product being evaluated explicitly. 


For discussion of the Stokes and Navier-Stokes equations, the reader is re- 
ferred to Deville (1984), Canuto et al. (1988), Fornberg (1996), and Gerritsma 
& Phillips (1998, 1999). 


11.10 Particular issues in spectral methods 


It is important to remember that the key advantages of spectral and pseu- 
dospectral methods lie in 


1. the rapid (e.g., exponential) convergence of the methods for very smooth 
data and PDEs, which makes Chebyshev methods so powerful; 


2. the use of discrete orthogonality, which greatly simplifies collocation 
equations; 
3. the use of the FFT, which speeds up computations typically from O(n?) 


to O(n log n) operations; 


4. the possibility of a matrix representation of derivatives, which simplifies 
the representation of the solution and boundary conditions in certain 
problems. 
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For the reasons above, the method will always be restricted to somewhat 
special classes of problems if it is to compete with more general methods like 
the finite element method. However, the spectral method shares with the 
finite element method a number of common features, including the pointwise 
and continuous representation of its solution (as in the differentiation matrix 
method) and the possibility of determining good preconditioners (Fornberg & 
Sloan 1994). 


We now raise some further important issues that arise in spectral /pseudo- 
spectral methods. We do not have the scope to illustrate these issues in detail 
but can at least make the reader aware of their significance. 


Aliasing (see Section 6.3.1) is an interesting feature of trigonometric and 
Chebyshev polynomials on discrete meshes. There is a potential for ambiguity 
of definition when a Chebyshev or Fourier series attempts to match a PDE 
on too coarse a grid. Fortunately, aliasing is not generally to be regarded 
as threatening, especially not in linear problems, but nonlinear problems do 
give cause for some concern on account of the possible occurrence of high- 
frequency modes which may be misinterpreted as low-frequency ones. Canuto 
et al. (1988, p.85) note that aliases may be removed by phase shifts, which 
can eliminate special relationships between low and high frequency modes. 


Preconditioners are frequently used in finite-element methods to improve 
the conditioning of linear equations. Their use with finite differences for Cheb- 
yshev methods is discussed for example by Fornberg (1996), Fornberg & Sloan 
(1994) and Phillips et al. (1986). The idea is, for example, to take a system of 
linear equations whose matrix is neither diagonally dominant nor symmetric, 
and to find a multiplying matrix that yields a result that is strictly diagonally 
dominant, and therefore amenable to Gauss-Seidel iteration. More broadly, 
it improves the conditioning of the system matrix. 


Basis functions in spectral methods may be not only Chebyshev polyno- 
mials, but also Legendre polynomials or trigonometric polynomials (Canuto 
et al. 1988). Legendre polynomials are sometimes preferred for Galerkin meth- 
ods and Chebyshev polynomials for collocation methods (because of discrete 
orthogonality). Trigonometric polynomials are versatile but normally suitable 
for periodic functions only, because of the Gibbs phenomenon (see page 118, 
footnote). Clearly we are primarily interested in Chebyshev polynomials here, 
and so shall leave discussion of Legendre polynomials and other possible bases 
to others. 


11.11 More advanced problems 


The subject of partial differential equations is a huge one, and we cannot in 
this broadly-based book do full credit to spectral and pseudospectral methods. 
We have chosen to illustrate some key aspects of the methods, mainly for linear 
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and quasilinear problems, and to emphasise some of the technical ideas that 
need to be exploited. 


For discussion of other problems and, in particular, more advanced PDE 
problems including nonlinear problems, the reader is referred to such books 


as: 


Canuto et al. (1988) — for many fluid problems of varying complexity 
and solution structures, as well as an abundance of background theory; 


Trefethen (2000) — for a very useful collection of software and an easy- 
to-read discussion of the spectral collocation approach; 


Boyd (2000) — for a modern treatment including many valuable results; 
Guo Ben-yu (1998) — for an up-to-date and very rigorous treatment; 


Fornberg (1996) — as it says, for a practical guide to pseudospectral 
methods; 


Deville (1984) — for a straightforward introduction mainly to fluid prob- 
lems; 


Gottlieb & Orszag (1977) — for an early and expository introduction 
to the spectral approach. 


11.12 Problems for Chapter 11 


1. 


Apply the method of separation of variables in (r, @) coordinates to 


gO Ot) Ore 
Tore Op + BGR 
(see (11.1la) above) in the disc S : r < 1, where u(1,0) = g(@) on 
OS :r =1, and g(@) is a known 27-periodic function of the orientation 
of a point P of the boundary. Determine the solution as a series in the 
cases in which 


Au =0 


. In addition to satisfying (m—1)(n—1) specified linear conditions in the 


interior of the square domain D: {|x| < 1, |y| < 1}, the form 


m+1 n+1 


So YS ay Tilx)Tiy) 


i=0 j=0 
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is collocated to a function g(x,y) at 2(m +n) points on its boundary 
OD. The latter points are chosen at the zeros of (1 — 2?)U;)—2(x) on 

= +1 and at the zeros of (1—y?)Un_2(y) on x = +1, where each of the 
four corners of the boundary (which occur in both sets of zeros) is only 
counted once. Investigate whether or not the resulting linear system is 
singular and determine its maximum rank. 


(This question is an analogue of a result in Section 11.8.5, where the 
zeros of T,,(a) and T;,(y) were adopted.) 


3. The diagram shows a square membrane with a slit from the midpoint 
A of one side to the centre O. We wish to determine solutions of the 
eigenvalue problem 


Aut Au = 0in S, 


u = O0on OS. 
E F 
S A 
O B 
D C 


Follow the style of Section 11.8.2 to transform the domain and problem 
into one which may be approximated by Chebyshev collocation. Use 
the mapping 2’ = z2 about O to straighten the cut AOB, determine 
equations for the mapped (curved) sides of the domain, determine the 
mapped PDE and deduce the form of approximation um,» to u. Describe 
the method of solution for A and u. 


[Note: The boundary equation is again y'(x?y? — r?2 +1) = 0 before 
mapping. | 


4. Investigate whether or not there is any gain of efficiency or accuracy in 
practice in using the Chebyshev form )° > ¢;;T2:41(v/a)T;(y/6) rather 
than )*> > ¢;v7"**ty? in the L-membrane approximation of Section 11.8.2 
and, similarly, for the relevant forms in the method for Problem 3 above. 
Is it possible, for example, to exploit discrete orthogonality in the col- 
location equations? 
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As a variant on the separation of variables method, consider the solution 
of 


Au = f(x,y) in the ellipse D : =) + RB <1, (A) 
ay? 
w= g(2,y) on OD: day) = +H =1, (B) 


where f = 0 and g is given explicitly on OD. 
Then the form 
Un = a0 + Y (ax coskO + by sin kO)r*, 
k=1 


where « = rcos@ and y = rsin48, satisfies (A) for all coefficients ag, ax, 
be. Compute ao,..., Qn, 01,...,0n so that (B) is collocated at 2n +1 
suitably chosen points of OD. It is suggested that equal angles of 0 
should be used on [0,27]; discuss some of the possible choices. What 
set of points would remain distinct as b — 0, if the ellipse has a small 
eccentricity? 


[Hint: Start at 0 = $7/(2n+ 1); the nodes for n = 2 are then chosen as 
in the figure and occur at 7/10, 57/10, 97/10, 137/10, 177/10. Choose 
simple non-polynomial functions for g; e.g., g(a, y) = cosh(x + y).] 


. Repeat Problem 5, but with g = 0 and f given explicitly on D, using 


the Chebyshev polynomial approximation 


se aig T; (y) 


j=0 


/ 


ot 


Umn = —s 


ll 
° 


t 


and collocating the PDE at a tensor product of the zeros of Ty,+1(«/a) 
and Tn+1(y/b) on the rectangle 


R:{-a<u<a;—b<y< bd}. 


Compute results for small choices of m, n. 


[Note: This is a method which might be extended to more general 
boundary ¢(z,y), and ¢ does not need to be a polynomial in z, y.] 


. Generate a set of tables similar to Tables 11.7—-11.11 for the odd/odd 


case m = n = 5, showing the 36 x 36 linear algebraic system for {a;,} 
and the four subsystems derived from this. 
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8. For m = n = 3 (see Section 11.8 above) show that the approximate 
solution Umn of (11.75) with g = 0, given by (11.91) with coefficients 
(11.119)—(11.122), may be simplified exactly into the form 


Umn = (1 —2”)(1 — y”)[a + ba + cy + day). 


What are the values of a, b,c, d? 
Derive Umn directly in this form by collocating the PDE at the Cheby- 


shev zeros. (Note that this method cannot be applied unless g(x,y) = 
0.) 


9. For m = n = 3 in (11.75), in the case where g is not identically zero, 
obtain formulae for the coefficients a;; in Umn from Tables 11.8-11.11, 
namely the four linear subsystems that define them. 


CHAPTER 12 


Conclusion 


In concluding a book that is at least as long as originally intended, we are 
painfully aware of all that we have left out, even though we have tried to 
include as much extra material in the Problem sets as we reasonably could. 
The reader is therefore referred to other texts for more advanced material, 
including the excellent book of Rivlin (1990), who includes fascinating ma- 
terial on various topics, such as the density of the locations of the zeros of 
Chebyshev polynomials of all degrees. We were also inspired by the book of 
Fox & Parker (1968), not only because it made the most up-to-date statement 
of its time but also, and even more, because it was extremely well written and 
stimulated major research. We hope that we are following in this tradition, 
and that there are numerous places in this book that future researchers might 
take as a starting point. 
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APPENDIX A 


Biographical Note 


PAFNUTY LVOVICH CHEBYSHEV was born in Okatovo in the Kaluga region 
of Russia on 16th May [4th May, Old Style] 1821. 


He studied mathematics at Moscow University from 1837 to 1846, then 
moved to St Petersburg (as it then was and has now again become), where 
he became an assistant professor at Petersburg University in 1847 and a full 
professor in 1851, in which post he remained until 1882. It is he who was 
principally responsible for founding, directing and inspiring the ‘Petersburg 
school’ of mathematical research, noted for its emphasis on drawing its prob- 
lems for study from practical necessities rather than from mere intellectual 
curiosity. He was elected a foreign associate of the Institut de France in 1874, 
and a fellow of the Royal Society of London in 1877. 


He worked in many fields outside approximation theory, including number 
theory (the distribution of primes), integration of algebraic functions, geomet- 
ric theory of hinge mechanisms (the subject which led to his special interest 
in minimax approximation of functions), the moment problem, quadrature 
formulae and probability theory (limit theorems). 


The Chebyshev polynomials T,, which now bear his name (the symbol ‘T” 
deriving from its continental transliterations as ‘Tchebycheff’, “I'schebyscheff’ 
&c.) were first introduced by him in a paper on hinge mechanisms (Chebyshev 
1854) presented to the St Petersburg Academy in 1853. They were discussed 
in more mathematical depth in a second paper (Chebyshev 1859) presented 
in 1857; see also (Chebyshev 1874). Somewhat surprisingly, in the light of 
what seems today the obvious connection with Fourier theory, his discussion 
makes no use of the substitution « = cos 6. 


He died in St Petersburg on 8th December [26th November, Old Style] 
1894. 


A much more extensive biography, from which these facts were extracted, 
is to be found in the Dictionary of Scientific Biography (Youschkevitch 1981). 
See also a recent article by Butzer & Jongmans (1999). 
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APPENDIX B 


Summary of Notations, Definitions and Important 
Properties 


B.1 Miscellaneous notations 


ye finite or infinite summation with first (To) term halved, 


YY arTy = $a0To + a1T1 + a2T2 ++ -- 
r=0 
Som finite summation with first and last terms halved, 


so" arly = 4aoTo + aT} a an—-1Tn-1 + 4anTn 
r=0 


ya finite summation with last term halved, 


yo ar Pr = a1 Py + +++ + Gn—1Pa—1 + $anPn 


r=1 


§ integral round a closed contour 


f Cauchy principal value integral 
[---] largest integer < --- 

\|- a norm (see page 43) 

(-,-) an inner product (see pages 72, 97) 


A(D) the linear space of functions analytic on the (complex) domain D 
and continuous on its closure D 


Bnf the minimax nth degree polynomial approximation to f on the 
interval [—1, 1] 


C{a,b] the linear space of functions continuous on the interval [a, 6] 


C”|a,b| the linear space of functions continuous and having n continuous 
derivatives on the interval [a, }] 


Cy the linear space of continuous periodic functions with period 27 
Crs the subspace of C9. consisting of even functions only 

C, the circular contour {w : |w| = r} in the complex plane 

D, the elliptic domain {z:1< |z + V2? — 1| <r} 

E, the elliptic contour {z: |z +722 — 1| =r} 
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APPENDIX B: SUMMARY OF NOTATIONS &C 


= the image of C,, under z = 3(w + w7!) 
the nth degree polynomial interpolating f at n+ 1 given points 


the linear space of functions on [a,b] on which the norm ||-||,, can 
be defined 


the linear space of polynomials of degree n 
the nth partial sum of the Fourier expansion of f 
the nth partial sum of the Fourier cosine expansion of f 


the nth partial sum of the Fourier sine expansion of f 


the nth partial sum of the first-kind Chebyshev expansion of f 
Lebesgue constant (see page 125) 
the modulus of continuity of a function (see page 119) 


the boundary of the two-dimensional domain S 
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B.2 The four kinds of Chebyshev polynomial 


Figure B.1: Plots of the four kinds of Chebyshev polynomial: T(x), Un(2), 
Vn(xz), W(x) for values of x in the range [—1,1] and n running from 0 to 6 
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APPENDIX C 


Tables of Coefficients 


Each of the following five Tables may be used in two ways, to 
give the coefficients of two different kinds of shifted or unshifted 
polynomials. 


Table C.1: Coefficients of a* in V,,(x) and of (—1)"t*a* in W,,(2) 


k=0 -1l -l 1 |) | 1 1 -1 -1 
1 2 —2 —-4 4 6 —6 —8 8 10 —10 
2 4 -4 -12 12 24 —24 —40 40 60 
3 8 —-8 -—32 32 80 —80 —160 160 
4 16 —16 —80 80 240 —240 —560 
5 32 —32 —192 192 672 —672 
6 64 —64 —448 448 1792 
A 128 —128 —1024 1024 
8 256 —256 —2304 
9 512) —512 

10 1024 
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Index 


Abramowitz, M., 305 
Achieser, N. I., 172, 305 
Adam, G., 186, 200, 305 
aliasing, 152-154, 193, 297 
alternation theorem 
for algebraic polynomials, 47 
for trigonometric polynomials, 
48 
Anderson, I. J., 96, 305 
approximation 
best, 45, 46 
good, 44 
least-squares, 76 
minimax, 47 
near-best, 45, 165-173 
near-best £1, 169-170 
near-minimax, 125, 160, 165— 
168 
Atkinson, K. E., 145, 146, 305 


Baltensprenger, R., 305 

Bayliss, A., 305 

Bennell, R. P., 96, 208, 214-217, 
221, 222, 305 

Bernardi, C., 305 

Bernstein, S. N., 172 

Berrut, J.-P., 305 

Berzins, M., 273, 306 

BIE, see integral equations, bound- 
ary 

Borel, E., 47, 306 

Boyd, J. P., 298, 306 

Branders, M., 186, 200, 316 

Brebbia, C. A., 222, 265, 306 

Brutman, L., vi, 161, 168, 206, 306 

Burrus, C. S., 92, 306 

Butzer, P. L., 306, 321 


Canuto, C., 93, 103, 271, 272, 295— 
298, 306 

Cauchy principal value integral, 182, 
209, 323 

Cebysev, see Chebyshev 


335 


Chalmers, B. L., 172, 173, 306, 314 
Chawla, M. M., 200, 306 
Chebyshev interpolation, see inter- 


polation 


Chebyshev norm, see norm 
Chebyshev polynomials 
1st kind (T,,) 


definition, 2, 12, 326 

derivatives, 34 

differential equation, 85 

discrete orthogonality, 85, 187, 
188, 328 

extreme values, 21, 326 

generating functions, 110-112 

integrals, 32 

minimal £, property, 158 

minimal £2 property, 78 

minimal £, property, 170— 
173 

minimax property, 49 

of complex variable, 14, 96 

orthogonality in complex plane, 
97, 327 

orthogonality over [—1, 1], 73, 
327 

recurrence relation, 2, 82, 326 

Rodrigues’ formula, 84 

zeros, 19, 326 


2nd kind (U;,) 


definition, 3, 12, 326 

differential equation, 85 

discrete orthogonality, 88, 189, 
328 

extreme values, 21 

integrals, 33 

minimal £, property, 158 

minimal £2 property, 78 

minimal £, property, 170— 
173 

minimax property, 50 

of complex variable, 15, 96 

orthogonality in complex plane, 
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97, 327 
orthogonality over [—1, 1], 74, 
327 
recurrence relation, 4, 82, 326 
Rodrigues’ formula, 84 
zeros, 20, 326 


3rd and 4th kinds (V,, Wn) 


definition, 5, 12, 326 

differential equation, 85 

discrete orthogonality, 89, 191, 
328 

extreme values, 21 

minimal £, property, 158 

minimal £2 property, 78 


INDEX 


compared to other orthogo- 
nal expansions, 128 
£2, 115 
pointwise, 118, 131 
uniform, 119 
examples, 105-110 
in complex domain, 138 
lacunary, 137 
multivariate, 120 
related to Fourier series, 113 
related to Laurent series, 140 
summation, 25 
with approximate quadrature 
equivalent to Chebyshev in- 


minimal £, property, 170— terpolation, 113, 150-152 
173 Chebyshev transform 
minimax property, 50 discrete, 90-91, 270 
of complex variable, 16, 96 Chebyshev, P. L., 306, 307, 321 
orthogonality in complex plane, life of, 321 
97, 327 Chebyshev—Padé approximations, 140 
orthogonality over [—1, 1], 74, Cheney, E. W., 46, 47, 124, 126, 


327 128, 147, 168, 307 
recurrence relation, 6, 82, 326 Chisholm, J. S. R., 140, 307 
Rodrigues’ formula, 85 Class, A., 305 


zeros, 20, 326 
four kinds (Tn, Un, Vn, Wn) 


Clenshaw, C. W., 24, 25, 28, 29, 
33, 95, 141, 180, 186, 261, 


graphs, 325 307 
summary of properties, 326— Clenshaw—Curtis quadrature, see quadra- 
328 ture 
tables of coefficients, 329- collocation method 
333 for eigenvalue equations, 252— 
generalised, 17 256, 274-277 
relationships between kinds of, for integral equations, 205 
7, 209 for ODEs, 234-237, 244-245, 
shifted, 57 249-250 


definition, 9 error of, 237-239 
Chebyshev semi-iterative method for PDEs, 269-271, 273-274, 

for linear algebraic equations, 279-281, 283-295 

52-55 for PDEs in space and time, 

Chebyshev series expansions 257-259 

1st kind, 105-133 Common, A. K., 140, 307 

2nd kind, 133 conditioning, 158 

3rd and 4th kinds, 135 continuity 

Cesaro summation, 119 Dini-Lipschitz, 119, 147, 162 

convergence modulus of, 119, 161 
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Cooley, J. W., 92, 261, 307 
Cox, M. G., 305 
Crampton, A., 283, 314 
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